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SYSTEM & APPARATUS FOR DISPLAYING SUBSTITUTE CONTENT 



CROSS-REFERENCE TO RELATED APPLICATION 

The present application is a continuation-in-part of 
and claims the benefit of U.S. Patent Application Serial 
No. 0 9/883,098, entitled "ADVERTISEMENT SWAPPING USING AN 
AGGREGATOR FOR AN INTERACTIVE TELEVISION SYSTEM' 7 by Douglas 
A. Miller, filed June 14 r 2001. Application 09/883,098 is 
fully incorporated herein by reference. 

TECHNICAL FIELD 

This disclosure relates generally to television 
programming content substitution methods and systems, and 
more particularly but not exclusively, to techniques for 

substituting programming content during commercial breaks 

or other breaks in a broadcast content. 

- BACKGROUND - : - - 

Conventionally, personal video recorders (PVRs) enable 
a viewer to record television programming content and then 
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to view the recorded television programming content at a 
later time. Further, PVRs may enable a viewer to record 
specified programming content on a regular basis and to 
delete the recorded programming content after a pre- 
specified amount of time. For example, a viewer may 
program a PVR to record a daily news show (such as the 5 
o'clock news) on a daily basis and then to automatically 
delete the recorded news show after one day since the news 
content would be stale. 

In order to view the recorded programming content, a 
viewer typically proceeds through multiple steps, which may 
include loading a user interface (UI) for display, 
selecting a "view pre-recorded programming content" option 
from a menu in the UI, and then selecting a pre-recorded 
programming content from a list of several pre-recorded 
programming content . 

PVRs also enable a viewer to pause "live" programming 
content (and/or buffered programming content) and to skip 
commercials and/or other portions of live television 
programming content (or buffered programming content) . For 
example, a viewer can skip commercials when viewing 
programming content on a television coupled to a PVR by 
pressing a "skip" button of the PVR remote control device. 

2 
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Thus, advertisers are concerned that the ability of 
viewers to skip commercials when viewing programming 
content with PVRs will lead to losses in revenue. 
Additionally, a viewer, while watching live TV may channel 
5 surf and lose the buffered portion of the content shown in 
the original channel. 

Accordingly, there is need for a new programming 
content substitution system and method that will overcome 
the deficiencies mentioned above. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Non-limiting and non-exhaustive embodiments of the 
present invention are described with reference to the 
following Figures, wherein like reference numerals refer to 
like parts throughout the various views unless otherwise 
specified. 

Figure 1 is a block diagram of an interactive video 
casting system that can implement an embodiment of the 
invention. 

Figure 2 is a block diagram of another interactive 
video casting system that can implement an embodiment of 
the invention. 

Figure 3 is a block diagram of a customer premise 
equipment, such as a set top box, according to an 
embodiment of the invention. 

Figure 4 is a block diagram of a memory device of the 
customer premise equipment of Figure 3. 

Figure 5A is a block diagram of a television screen 
where the substitute content is shown as a full screen view 
across the television screen. 

Figure 5B is a block diagram of a television screen 
where the substitute content is shown as the main screen 
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view, while the programming content is shown as a picture- 
in-picture. 

Figure 5C is a block diagram of a television screen 
where the substitute content is shown on the television 
5 screen foreground, while the programming content is shown 
on the television screen background. 

Figure 5D is a block diagram of a television screen 
where the substitute content is shown on one portion of a 
split screen view, and the programming content is shown on 
10 another portion of the split screen view. 

Figure 6A is a block diagram of a programming content 
that includes a disable feature to prevent disruption of 
the viewing of commercial segments in the programming 
content . 

15 Figure 6B is a flowchart of a method of preventing the 

viewing of substitute content during program breaks. 

Figure 6C is a flowchart of a method of permitting a 
user to view substitute content during program breaks. 

Figure 7 is a flowchart of a method for substituting a 
20 live or pre-recorded (buffered) programming content with 

substitute content, in accordance with an embodiment of the 
invention. 
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Figure 8 is a block diagram of another representative 
customer premise equipment, such as a set top box, 
according to an embodiment of the invention; 

Figure 9 is a block diagram of a memory device of the 
5 customer premise equipment of Figure 8 . 

Figure 10 is a flowchart of a method for substituting 
a live or other programming content with trigger-associated 
information, in accordance with an embodiment of the 
invention. 

10 Figure 11A is a block diagram of another interactive 

television system that can implement an embodiment of the 
invention. 

Figure 11B is a block diagram of a system for 
permitting sharing of aggregated substitute content. 
15 , Figure 12 is a flowchart of a method of collecting and 
displaying aggregated substitute content, in accordance 
with an embodiment of the invention. 
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DETAILED DESCRIPTION OF THE ILLUSTRATED EMBODIMENTS 

Embodiments of a system and method for substituting a 
live or pre-recorded (or buffered) programming content 
(i.e., broadcast content) with a substitute content are 

5 disclosed herein. The substitute content may be, for 

example, a viewer-selected content or default content. As 
an overview, in an embodiment, a broadcast content 
substitution system includes a customer premise equipment 
(or other processor) that can permit the display of 
10 broadcast content. In response to the occurrence of a 
break in the broadcast content, the customer premise 
equipment can permit the display of substitute content. 
The break may be, for example, a commercial in a live 
broadcast content, a commercial skip segment in a buffered 

15 live broadcast content, a broadcast stoppage, or a non- 
activity in a broadcast event. When the break in the 
broadcast content terminates, the customer premise 
equipment can again permit the display of the broadcast 
content. During the occurrence of the break, the customer 

20 premise equipment may permit the display of the substitute 
content and images related to the break in the broadcast 
content . 

In another embodiment, the invention broadly provides 
a customer premise equipment (or other processing device) 
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that includes a recording engine and a playback engine . 
The recording engine enables a viewer to select programming 
content to record on, for example, a regular basis, which 
is sometimes referred to as a "season's pass." For 
example, a viewer can program the recording engine to 
record a 5 o'clock news show on a daily basis. The 
recording engine may also automatically delete recorded 
programming content after a selected amount of time. 

The playback engine, upon receiving a user-enabled 
command signal, can substitute the substitute content in 
place of the currently-displayed live programming content 
or buffered programming content. In another embodiment, 
commercial breaks in the currently-displayed live 
programming content or buffered programming content are 
automatically detected, and the substitute content is 
automatically displayed during the commercial breaks. The 
substitute content can be a previously-recorded programming 
content, a live programming content, or content from an 
Internet Protocol (IP) channel, such as streaming video, 
electronic mail interaction, and so on. The electronic 
mail interaction would be automatically activated during, 
for example, commercial breaks and de-activated (or 
paused/halted) during the non-commercial segments in the 
live programming content or buffered programming content. 

8 
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The substitute content may be specified by the user or may 
be a default content. 

In another embodiment, the substitute content is 
displayed during breaks in a video telecast or 

5 teleconference (where a teleconference may be, for example, 
a Remote Learning Session or Video Conference) . A break in 
a live action in a video telecast or teleconference may 
occur, for example, when there is a change in speakers, 
during the time when equipment is .set up, during unforeseen 
10 delays in the presentation, and so on. 

The substitute content may, for example, be displayed 
in full screen mode on a television. Alternatively or in 
addition, the substitute content may be displayed as the 
main screen view, while the live or other programming 

15 content is displayed as a picture-in-picture on the 

television screen. Alternatively or in addition, the 
substitute content may be displayed as a picture overlay on 
the television screen foreground, while the live or other 
programming content is displayed on the television screen 

20 background. Alternatively or in addition, the substitute 
content and the live or other programming content may be 
displayed in separate screen areas as split screen views on 
the television screen. 

9 
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In another embodiment, the invention broadly provides 
a customer premise equipment (or other processing device) 
that includes a trigger marking engine and a trigger 
viewing engine. The customer premise equipment can receive 
programming content that includes Advanced Television 
Enhancement Forum (ATVEF) triggers or other types of 
triggers. One type of trigger, ATVEF B, may also be used 
and includes content as well as triggers. If a viewer is 
interested in a particular trigger that appears on the 
television screen, then the viewer can, for example, enable 
a command to signal the customer premise equipment to mark 
a Uniform Resource Locator (URL) address (or other address 
information peculiar to the Internet or other data 
communication network) where the address is associated with 
the trigger of interest. The URL address or other address 
is then stored for later viewing and access. During a 
commercial or other breaks in a live or buffered broadcast 
content, the viewer can then signal the customer premise 
equipment to show and access the stored URL address (or 
other address) in order to view information associated with 
previous triggers of interest. When a website (or other 
information) associated with the accessed URL address (or 
other address) is shown, the website may, for example, be 
shown in full screen on the television. Alternatively or 
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in addition, the website may be displayed on the television 
screen, while the live or buffered broadcast content is 
displayed as a picture-in-picture on the television screen. 
Alternatively or in addition, the website may be displayed 
5 as a picture overlay on the television screen foreground, 
while the live or buffered broadcast content is displayed 
on the television screen background. Alternatively or in 
addition, the website and the live or buffered broadcast 
content may be displayed in separate areas as split screen 

10 views on the television screen. The viewer can then, for 
example, obtain information or buy items or services in the 
displayed website. The viewer may then return to viewing 
the live or buffered broadcast content by enabling an 
appropriate command, for example, via the remote control 

15 device. 

In another embodiment of the invention, an aggregator 
can be used to aggregate user-selected content. The 
aggregated user-selected content can then be provided for 
playback to the user, for example, during commercial breaks 
20 in a live or buffered programming content or during breaks 
in a video telecast or teleconference. For example, the 
aggregator can be used to locate and collect programming 
content (or portions of a programming content) related to a 
specific topic as specified by the user. The collected 

11 
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programming content can then be displayed during commercial 
breaks in a live or buffered broadcast or during breaks in 
a video telecast or teleconference. In one embodiment, the 
aggregator can provide the aggregated user-selected content 
to a storage unit (such as a PVR) for subsequent playback. 
In another embodiment, a customer premise equipment (such 
as a set top box) can perform channel switching to switch 
from a channel where the live or buffered programming 
content is showing to another channel (e.g., a dedicated 
broadcast channel) where the aggregated user-selected 
content can be shown. 

In the description herein, numerous specific details 
are provided, such as examples of system components and/or 
methods, to provide a thorough understanding of embodiments 
of the invention. One skilled in the relevant art will 
recognize, however, that the invention can be practiced 
without one or more of the specific details, or with other 
systems, methods, components, materials, parts, and/or the 
like. In other instances, well-known structures, 
materials, or operations are not shown or described in 
detail to avoid obscuring aspects of the invention. 

Reference throughout this specification to "one 
embodiment", "an embodiment", or "a specific embodiment" 
means that a particular feature, structure, or 

12 
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characteristic described in connection with the embodiment 
is included in at least one embodiment of the present 
invention. Thus, the appearances of the phrases "in one 
embodiment' 7 , "in an embodiment ", or "in a specific 

5 embodiment" in various places throughout this specification 
are not necessarily all referring to the same embodiment. 
Furthermore, the particular features, structures, or 
characteristics may be combined in any suitable manner in 
one or more embodiments. 

10 Additionally, the signal arrows in the 

drawings/Figures are considered as exemplary and are not 
limiting, unless otherwise specifically noted. 
Furthermore, the term "or" as used in this disclosure is 
generally intended to mean "and/or" unless otherwise 

15 indicated. Combinations of components or steps will also 

be considered as being noted, where terminology is foreseen 
as rendering the ability to separate or combine is unclear. 

It will also be appreciated that one or more of the 
elements depicted in the drawings/figures can also be 

20 implemented in a more separated or integrated manner, or 

even removed or rendered as inoperable in certain cases, as 
is useful in accordance with a particular" application. 

Figure 1 is a block diagram of an interactive video 
casting system 100 that can implement a specific embodiment 

13 
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of the invention. Of course, the illustrated system 100 is 
provided for example purposes only, and other systems or 
topologies may be used within the scope of the invention. 
In an embodiment, a production company 104 produces 
programming content for transmission to viewers. The 
transmission is sent via a satellite transmission 
transceiver 112 over an uplink channel to a satellite 102. 
The satellite 102 then transmits the programming content 
over a downlink channel to a local studio 106. The- local 
studio 106 can insert additional programming content (e.g., 
regional programming content) and/or advertisements as 
needed into the programming content. The programming 
content with the insertions is then transmitted from the 
local studio 106 to a cable service provider 108. The 
television programming content may be downloaded to a 
receiving station, such as a broadcast center (e.g., cable 
distribution head-end) (not shown) of the cable service 
provider 108, rather than or in addition to the local 
studio 106. A reverse channel from the cable service 
provider 108 to the local studio 106 may be provided so 
that the local studio 106 can insert additional programming 
content and feed the television signal back to the cable 
service provider 108. The cable service provider 108 then 
delivers the television signal over a cable network 134 to 

14 
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cable subscribers . 

The cable network 134 is provided by the cable service 
provider 108 to distribute the programming content to cable 
subscribers. A set top box (STB) 152, typically located on 
5 the premises of a cable television subscriber, receives the 
programming content or television signal from the cable net 
134, and delivers the television signal to the subscriber's 
television set (or other display device) 154. In some 
specific embodiments, alternatively or in addition, the 

10 television signal can be broadcast over a wireless medium 
and received by a traditional aerial antenna or by a 
satellite dish, and then delivered to the set top box 152. 
Alternatively or additionally, features and functionality 
of the set top box 152 may be integrated into a type of 

15 advanced television or other display device. 

Moreover, embodiments of the invention can use other 
types of broadcast media, including but not limited to, 
digital cable systems, satellite, very-high-data-rate 
digital subscriber line (VDSL), web casts, and the like. 
20 The features provided by the television set 154 can also be 
provisioned, in one embodiment, by a personal computer (PC) 
suitably configured with an adapter to convert television 
signals into a digitized format, and then to deliver the 

15 
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television signals to the video portion of the computer for 
display. It is 'noted that the invention is not limited to 
any one configuration of display hardware as embodiments of 
the invention can be realized using alternative reception 
and display arrangements, as known to those skilled in the 
art . 

In accordance with an embodiment of the invention, a 
connection to a communication network is provided for the 
cable subscriber. In one embodiment, the connection can be 
made via a cable modem 156 over a bi-directional 
communication link 155 to a cable modem termination system 
within the cable provider's 108 equipment. The connection 
continues to a data communication network, such as the 
Internet, by way of a public switched telephone network 
(PSTN) 132. The PSTN network 132 is provided herein as an 
example, and it is understood that other types of networks 
may be used for connectivity to the Internet. A cable 
modem arrangement can be used because of its high bandwidth 
capability. In situations where some cable companies are 
not equipped to provide cable modem service to their 
customers, various other arrangements can be made. For 
example, a conventional modem connection can be used to 
access the Internet over a telephone line. As another 
example, Internet access can be gained over a DSL 

16 

02104037A1J_> 



WO 02/104037 

■ 1 



PCT/US02/09148 



connection or an integrated services digital network (ISDN) 
connection using a telephone line. Wirqless systems are 
also available for providing Internet access. In one 
embodiment , downstream data transmission may occur via 
5 cable or satellite, and upstream data transmission may 
occur via a telephone line. 

It is noted that the Internet is chosen as an example 
of a data communication network because it is a well- 
established network, and connectivity to the Internet is 
10 easily made. However, it is noted that a global 

communication network, such as the Internet, is not 
required to practice other embodiments of the invention. A 
locally provided and maintained communication network may 
be used in an embodiment of the invention. 

15 Continuing with Figure 1, the set top box 152 can 

include a transceiver 157, such as an infrared (IR) or 
radio frequency (RF) transceiver, that can exchange signals 
with a remote control unit 158 or other user input device. 
The set top box 152 can be a component that is separate 

20 from the television set 154 as shown in Figure 1, or its 

features can be built into circuitry of the television set 
154 (e.g., an interactive television set). The set top box 
152 enables a viewer to select television programming 

17 
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content to view and then delivers the television 
programming content to the television set 154. A storage 
unit 162 can also be coupled to or be a part of the set top 
box 152. The storage unit 162 can include a machine- 
readable storage medium such as a cache, buffer, memory, 
diskette, compact disk, tape, or the like and their 
associated hardware, in one embodiment. In another 
embodiment, the storage unit 162 can include a video 
cassette recorder (VCR) . In another embodiment, the 
storage unit can include a hard disk such as a digital or 
personal video recorder (DVR or PVR) . 

As noted above, the local studio 106 can insert 
additional programming content into the received 
transmission, for example, to provide cable content that 
includes locally provided channels. The programming 
content is then distributed to customers over the cable 
network 134. In addition to local programming content 
insertion, the local studio 106 can insert advertising 
content. Product supplemental information relating to the 
advertising for participating merchants 122 can also be 
inserted. Product supplemental information can include 
information relating to the goods or services being 
advertised in the commercial. In addition to goods and 
services, coupons and other information services can be 

18 
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made available to the viewer, which in one embodiment can 
be obtained via the merchant's 122 web site 124 on the 
Internet. Triggers, such as Advanced Television 
Enhancement Forum (ATVEF) triggers, which are related to 
the web site 124 and/or to its contents, can be 
continuously updated as the television broadcast is being 
received. 

As noted above, the triggers, resources, or 
announcements can be inserted by the originating 
broadcaster 104, a local broadcaster 106, or by the cable 
system operator 108. 

Figure 2 is another block diagram showing an example 
of an interactive video casting system 200 and illustrates 
another point of insertion of the product supplemental 
information. In a specific embodiment shown in Figure 2, a 
merchant 222, operating a web site 224, is located such 
that the merchant 222 is capable of inserting commercial 
prior to the uplink transmission to the satellite 102. 

As noted above, Internet access is not necessary to 
practice the invention. A locally provided network may, 
for .example,, be used to practice _an ^embodiment, of_ithe . 
invention. The cable provider 108 can supply the foregoing 
features, for example, by providing a web site or "walled 

19 
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garden" that is accessed by its subscribers. In such a 
case, the cable provider 108 serves as an intermediary and 
submits the purchases to the actual merchants 122 or 222. 

In another embodiment, programming content and/or 
other data may be distributed to a subscriber/customer via 
a satellite TV delivery system, which may include a direct 
broadcast satellite (DBS) system. A DBS system may 
include, for example, a small 18-inch satellite dish (which 
is an antenna that can receive a satellite broadcast 
signal); a digital integrated receiver/decoder (IRD) , which 
separates each channel, and decompresses and translates the 
digital signal so that a television can show the digital 
signal; and a remote control that transmits command 
signals. Programming for a DBS system may be distributed 
by, for example, multiple high-powered satellites in geo- 
synchronous orbit, with each satellite having multiple 
transponders. Compression (e.g., Moving Pictures Experts 
Group or MPEG based compression) may be used to increase 
the amount of programming content that can be transmitted 
in the available bandwidth. 

A digital broadcast center may be used to gather 
programming content, ensure the digital quality of the 
programming content, and transmit the signal up to the 
satellites for delivery. Programming content may be 
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provided to the broadcast center from content providers 
(e.g., ABC, CNN, ESPN, and so on) via satellite, fiber 
optic cable, and/or special digital tape. Satellite- 
delivered programming content are typically immediately 
5 digitized, encrypted, and up-linked to the orbiting 

satellites. The satellites re-transmit the signal back 
down to every earth-station (or, in other words, every 
compatible DBS system receiver dish at, for example, homes 
and businesses of customers) . 

10 Some programming content may be recorded on digital 

videotape in the broadcast center, and the programming 
content can be broadcast at later time or day. Before any 
recorded programming content are viewed by customers, 
technicians may use post-production equipment to view and 

15 analyze each tape to, for example, ensure audio and video 
quality. Tapes may then be loaded into, for example, a 
robotic tape handling system, and playback may be triggered 
by a computerized signal sent from a broadcast automation 
system. Back-up videotape playback equipment may ensure 

20 uninterrupted transmission at all times. 

Figure 3 illustrates a block diagram of a 
representative set top- box (or other suitable customer 
premise equipment or processing device) 152a according to a 
specific embodiment of the invention. It is noted that the 

21 
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STB 152a described below is one of the possible embodiments 
of the STB 152 that is shown in Figure 1 and Figure 2. It 
is further noted that the components shown in Figure 3 may 
be implemented in other types of customer premise equipment 
or other types of processing devices. In an embodiment, 
the STB 152a includes a network interface 300, processor 
310, memory device 330a, transceiver 157, converter 320, 
cable modem 156, optional clock 350, and hard disk 162 
(which may be any suitable storage device) , all 
interconnected together for communication via a system bus 
340. For purposes of explaining the functionality of the 
invention, other known components or modules or software in 
a set top box have been omitted in Figure 3. Network 
interface 300 connects the STB 152a to the cable network 
134 (Figure 1) . In an alternative embodiment, the cable 
modem 156 or the converter 320 may provide some or all of 
the functionality of the network interface 300, and thus, 
the network interface 300 may be omitted. In another 
embodiment, the network interface 300 may also provide some 
or all of the functionalities of the converter 320 and 
cable modem 156, and as a result, the converter 320 and/or 
cable modem 156 may be omitted in this particular 
embodiment. Thus, the channels shown in Figure 3 (e.g., 
channel 322) may be coupled to the cable modem 156 and/or 

22 



02104037A1I > 



WO 02/104037 



PCT/US02/09148 



to the network interface 300. 

The processor (controller) 310 executes instructions 
stored in memory 330a, which will be discussed in further 
detail in conjunction with Figure 4. The processor 310 
5 also typically performs other management functions of the 
STB 152a. 

The transceiver 157 can include an infrared (IR) or 
radio frequency (RF) transceiver (or other transceiver 
types) that can exchange signals with the remote control 
10 unit 158 (Figure 1 or Figure 2) or other user input device. 
The converter 320 can convert, if necessary, digitally 
encoded broadcasts to a format usable by the television 154 
(Figure 1 or Figure 2) . In addition, the converter 320 can 
convert other data received in an out-of-band portion of a 
15 broadcast, such as television scheduling information to a 
format that can be stored in the memory 330a or hard disk 
162. 

The cable modem 156 can transmit and receive digital 
information, such as television scheduling information, if 
20 not included in the out-of-band portion of a broadcast. In 

alternative embodiments, the cable modem 156 may be a 

conventional modem for use over telephone lines or may 
include any other component or module for transmitting and 
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receiving digital data. 

The STB 152a may include the hard disk 162 or, in an 
alternative embodiment, the hard disk 162 may be external 
to STB 152a. For example, in one embodiment, a PVR (not 
shown) may include the hard disk 162 and may be coupled to 
the STB 152a. The hard disk 162 may store television 
programming content in MPEG format or other format or 
combination of formats. 

The optional clock 350 is typically a digital clock 
for use with the engines that can be stored in the memory 
device 330a (see Figure 4). The clock 350 can be updated 
regularly by using the cable modem 156 to contact an online 
clock, such as the Atomic Clock at the U.S. Naval 
Observatory, or to contact a head-end at the provider 108 
so that the clock 350 is synchronized with the clock of 
provider 108. Alternatively, the clock 350 may be set 
manually by a viewer or may be updated through other 
methods known to those skilled in the art. 

Figure 4 is a block diagram of a memory device 330a in 
the STB 152a (Figure 3) . In the specific embodiment shown 
in Figure 4, the memory device 330a includes a recording 
engine 410; recording data 420; playback engine (or viewing 
engine) 430; playback data 440; and applications 445. The 
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various engines discussed herein may be, for example, 
software, commands, data files, programs, code, modules, 
instructions, or the like. It is also understood that the 
various engines described herein may be combined. For 
5 example, the recording engine 410 and playback engine 430 
may be combined as a substitute content engine 432, 
Furthermore, the various engines (e.g., applications 445) 
described herein may reside in other storage devices such 
as hard disk 162. 

10 The recording engine 410 may enable a viewer to 

schedule a "season's pass'' recording schedule, i.e., to 
record specific programming content on a daily or other 
regularly scheduled basis. The recording engine 410 may 
also enable a viewer to schedule the recording of a 

15 particular programming content at a specific day/time. The 
programming content may be received by the STB 152a via, 
for example, the cable net 134. The recording engine 410 
then can record the programming content onto the hard disk 
162 in MPEG format or other format for later viewing on the 

20 television 154. 

The recording data 420 includes viewer-specified 
recording instructions for the recording engine 410, such 
as the particular channel (s) to record, the time/day to 
start recording, and the length of the recording time 
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and/or the time to stop recording. The playback engine 4 30 
sends the recorded programming content to the converter 320 
to convert to a format capable to be displayed on the 
television 154 , if necessary. The playback engine 430 may 
5 also convert the programming content into a format 
displayable by the television 154 depending on the 
capabilities of the converter 320. 

In one embodiment, upon receiving a user-enabled 
command signal 426 (e.g., from the user-controlled remote 
10 control device 158), the playback engine 430 can set the 
substitute content 4 42 (see Figure 5A) in place of the 
currently displayed live programming content or buffered 
programming content as shown on the screen 444 of the 
television 154. Of course, the user can enable the command 
15 signal 42 6 by directly actuating a button or switch on the 
set top box 152a, instead of using, for example, the remote 
control device 158. In another embodiment, commercial 
breaks in the currently displayed live or buffered 
programming content are automatically detected by the 
20 playback engine 430, and the substitute content 442 is 

automatically displayed during the commercial breaks and 
paused (or removed from the screen 4 44) when the commercial 
break ends and the main program in the broadcast resumes. 
Commercial breaks in the live or buffered programming 
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content , can be detected by having the set top box 152a 
monitor for triggers or tags that demarcate commercial 
breaks . 

More than one substitute content 442 may be displayed 
5 by the playback engine 430. The substitute content 442 can 
be a previously-recorded programming content (e.g.', 
advertisements, news, movies, and/or other types of shows) 
as recorded by the recording engine 410, live programming 
content as selected on a particular channel by the playback 
10 engine 430, or content from an Internet Protocol channel, 
such as streaming video, electronic mail interaction, and 
so on. The electronic mail interaction would be 
automatically activated during, for example, commercial 
breaks and de-activated (or halted) during the non- 
15 commercial segments in the live or buffered programming 
content. The substitute content 442 may be, for example, 
stored content as specified by the user, may come from a 
selected channel as specified by the user, may be content 
from a default channel, and/or may be stored content that 
20 is chosen by default by the playback engine 4 30. As 
described below with reference to Figure 11A, the 
substitute content 442 may also be content from- an— ■ 
aggregated content collection that is based upon a criteria 
selected. by the viewer. 
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In another embodiment, the substitute content 442 is 
displayed for a particular amount of time (e.g., 30 
seconds) by the playback engine 430 after a commercial 
break occurs in the programming content 446 currently 
5 displayed on the television screen 444 (see, e.g., Figure 
5C) . 

In another embodiment, the substitute content 442 is 
displayed during breaks in a video telecast or 
teleconference (where a teleconference may be, for example, 
10 a Remote Learning Session or Video Conference) that are 
shown in the television screen 444 . A break in a live 
action in a video telecast or teleconference may occur, for 
example, when there is a change in speakers, during the 
time when equipment is set up, during unforeseen delays in 
15 the presentation, during broadcast stoppage, and so on. 

Thus, an embodiment of the invention advantageously 
permits the user to view one or more substitute content 442 
on the television screen 444 during commercials and/or 
other breaks (e.g., broadcast stoppage) in a currently- 
20 shown live broadcast content or buffered broadcast content. 
The user can, therefore, optimize his/her viewing time by 
viewing other content when a break occurs in a currently- 
shown live broadcast content or buffered broadcast content. 
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An embodiment of the invention also advantageously 
enables an alternative medium for showing advertisements, 
since the substitute content 442 can also include 
advertisements. The advertisements in the substitute 
content 4 42 can be shown during commercial breaks of a 
currently shown broadcast content, during the skip segment 
(e.g., a 30-second skip segment) of a buffered broadcast 
where the skip segment contains recorded advertisements in 
the buffered broadcast, or during other breaks in a current 
broadcast content . 

The playback engine 4 30 may also have the 
functionality of a display engine for showing the images on 
the screen 444 (see Figure 5A) of the television 154 in 
various formats. For example, the substitute content 4 42 
may be displayed in full screen mode on the television 
screen 4 44, as shown in Figure 5A, while the current 
programming content is removed from the screen 444 . 

Alternatively or in addition, as shown in Figure 5B, 
the playback engine 430 may display the substitute content 
442 as the main screen 444 view, while displaying the live 
or other programming content 446 as a picture-in-picture 
448 on the television screen 444. Of course, the 
substitute content 442 may alternatively be displayed in 
the picture-in-picture 448, while the programming content 
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448 is displayed as the main screen 4 44 view. 

■ Alternatively or in addition, as shown in Figure 5C, 
the substitute content 4 42 may be displayed as a picture 
overlay on the television screen 444 foreground, while the 
5 live or other programming content 44 6 is displayed on the 
television screen 444 background. 

Alternatively or in addition, as shown in Figure 5D, 
the substitute content 442 and the live or other 
programming content 446 may be displayed in separate areas 
10 448 and 450, respectively, as, for example, split screen 
views on the television screen 444. 

Prior to sending the substitute content 4 42 from the 
hard disk 162 (or via a selected broadcast channel as 
graphically represented by line 322)*, the substitute 
15 content 442 is typically decoded by the converter 320 

(Figure 3) prior to displaying the substitute content 4 42 
on the television screen 444. Similarly, prior to 
displaying the programming content 44 6 on the television 
screen 44 4, the programming content 4 46 is typically 
20 decoded by the converter 320. 

Referring again to Figure 4, the memory 330a may also 
store applications 445 which may represent one or more 
application. The applications 44 5 may also be stored in 
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the digital storage device 162. The applications 445 may 
include, for example, a browser for accessing a Uniform 
Resource Locator (URL) address on the Internet and for 
permitting web pages in a URL address to be displayed on 
5 the television screen 444. The applications 445 may also 
include an e-mail engine that can detect when an e-mail 
message has been received by the set top box 152a. The 
email engine can also permit the display of an e-mail 
message on the television screen 4 44 f the creation and 
10 transmission of an e-mail message from the STB 152a, and/or 
other e-mail processing functions. The applications 445 
may also include a streaming video engine for processing 
streaming video that is received from an IP channel. 

The applications 445 may also include middleware 
15 (sometimes called "plumbing") for connecting two sides of 
an application and passing data between them. 

The playback data 440 may be stored in, for example, 
the hard disk 162 and include a list of recorded substitute 
content 442 (e.g., substitute contents 442a and 442b. The 
20 playback engine 430 determines the user-selected substitute 
content 4 42 to be displayed on the television screen 44 4 in 
response to a user-enabled command 426 (e.g., provided by a 
user via a remote control device) or in response to the 
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detection of a break in the currently-shown programming 
content 446. 

The playback data 440 may also include a tag 451 
(Figure 4) identifying a specific recorded substitute 
content to be displayed as the default substitute content 
in response to user-enabled commands 426 or in response to 
the detection of a break in the currently-shown programming 
content 44 6. Thus, the default substitute content may also 
be displayed automatically when a commercial break is 
detected in the currently displayed programming content. 
Detection of commercial breaks in a programming content is 
performed by, for example, detecting identifiers that 
identify the beginning and end of a commercial. Various 
methods or mechanisms can be used to identify a sequence, 
such as, for example, include MPEG methods that multiplex 
special data stream (s) into the digital video transport 
stream, encoding/embedding data in the vertical blanking 
interval (VBI) portion of the analog television video 
signals (e.g., ATVEF triggers), using out-of-band modems 
(e.g., Digital Audio-Visual Council or "DAVIC" modems) to 
provide the interactive content, and other such techniques 
that provide information related to the content of the 
television signals. Another method to identify breaks in 
the programming content is by scanning the closed caption 
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text in the programming content for keywords that are of 
interest to the user. This method can be used to determine 
the beginning and ending of break segments. For example, a 
local new broadcast can be broken into "headlines", 
5 prominent local stories, national news, international news, 
business news, human interest stories, sports, weather, and 
commentary. A user may elect to skip the sports or weather 
section (especially an interactive user that receives these 
content on-demand) and to use this segment time to view 
10 buffered content (i.e., substitute content). 

Further, the playback data 4 40 may also include tags 
452 indicating the portions of a recorded substitute 
content that have been shown by the playback engine 430 for 
viewing on the television screen 444. The tags 4 52 thus* 

15 permits the playback engine 4 30 to skip the showing of 

previously shown substitute content. Thus, as an example, 
the user may be viewing a first program (e.g., programming 
content 446) on the screen 444. When a commercial break 
occurs in the first program, the end user may initiate a 

20 second program (e.g., substitute content 442) to be 

displayed on the televi3ion screen 444. As discussed 
above, the user may reduce . the first program into, for 
example, a picture-in-picture 448 and bring the second 
program on the main window of screen 4 44. When the 
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commercial break in the first program comes to an end, the 
user can perform a swap function to bring the first program 
from the picture-in-picture 448 to the primary focus on the 
screen 444. In one embodiment, the second program will 

5 pause, and the tags 4 52 can be placed by the playback 
engine 430 (Figure 4) at the segment where the second 
program is paused. The user can then resume viewing of the 
second program (based on the tag 452 location) during the 
next set of commercials. Of course, other types of pause 

10 features or techniques may be used to indicate where the 
second program (substitute content) is paused. 

In one embodiment, a live programming content (such as 
content 446) may also be continuously recorded (e.g., in a 
live program buffer in hard disk 162) while the viewer is 

15 viewing the substitute content (e.g., substitute content 
4 42) . Thus, if a commercial break occurs in the live 
programming content, the viewer can switch viewing to the 
substitute content while the recording engine 410 records 
the live programming content. The live programming content 

20 may be, for example, recorded automatically (as programmed 
by the user) or upon receiving user-enabled commands from, 
for example, a user-controlled remote control device 158. 
If the commercial break (in the live programming content) 
ends before the viewer finishes watching at least a segment 
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of the substitute content 4 42, then the following 
compensation may occur. When the viewer switches viewing 
from the substitute content back to the programming content 
4 4 6 , the user ca; A , for example, issue via remote control 
5 device 158 a replay or rewind command so that the recorded 
portion of the live programming content is rewound to 
permit the user the opportunity to view the missed portions 
of the live programming content. Since the live 
programming content may be buffered in a live program 
10 buffer, the viewer can view the remaining segments of the 

live programming content after watching the missed recorded 
portions of the live programming content. Alternatively or 
in addition, the live program can be paused after the end 
of the commercial segment. 

15 Figure 6A is a block diagram of a programming content 

4 4 6a with a disable feature to prevent the disruption of 
viewing of commercial segments in the programming content 
446a. For example, an advertiser can pay the broadcaster 
extra fees (or rates) on the rate card to insert tags 455 

20 prior to a commercial segment 457. The tag 455 deactivates 
the playback engine 4 30 so that the viewer will not be able 
to view the substitute content 4 42 when the commercial 
segment 457 is showing on the television screen 444. As a 
result, an advertiser can pay extra rates to the 
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broadcaster to insure the viewing of the commercials of the. 
advertisers to consumers. 

Figure 6B is a flowchart diagram illustrating a method 
4 60 of preventing the disruption of viewing of commercial 
5 segments in the programming content 4 4 6a. An advertiser or 
broadcaster (or other party) may pay (462) a multiple 
system operator (MSO) a fee or extra rate to prevent 
disruption of a commercial in the broadcast content. The 
fee or extra rate enables triggers or tags that prevent s 
10 subscriber to view substitute content during a commercial 
break in the broadcast content. The advertiser (or 
broadcaster or other party) transmits (464) a trigger that 
prevents the display of substitute content during a 
commercial break in the broadcast content. For example , ■ 
15 the trigger or tag may prevent an overlay with substitute 
content from being displayed during the commercial break. 
As a result, the display of a substitute content is 
prevented (466) during a commercial break. The method 460 
permits the MSO to obtain additional revenues from the fee 
20 (or extra rate) paid by an advertiser (or broadcaster) , and 
permits an advertiser to prevent disruptions of 
commercials . 

Figure 6C is a flowchart diagram illustrating a method 
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470 of permitting a subscriber to view substitute content 
during a commercial segment in the programming content 
4 4 6a. A subscriber may pay (472) a multiple system 
operator (MSO) a fee or extra rate to permit the subscriber 

5 to view substitute content during a commercial in the 

broadcast content. The advertiser (or broadcaster or other 
party) transmits (474) a trigger that prevents the display 
of substitute content during a commercial break in the 
broadcast content. For example, the trigger or tag may 

10 prevent an overlay with substitute content from being 

displayed during the commercial break. The subscriber can 
disable (476) the trigger or tag that prevents the display 
of substitute content. As a result , the subscriber may 
view (478) a substitute during a commercial break. The 

15 method 470 permits the MSO to obtain additional revenues 

from the fee (or extra rate) paid subscriber, and permits a 
subscriber to view substitute content during commercial 
breaks. 

Thus, the tags 4 55 (Figure 6A) may work in cooperation 
20 with an MSO and/or subscriber authorization features to 
enable/disable the effectiveness of the tags 455. As 
mentioned above, a subscriber who pays an additional fee 
may be able to disable the. tags 4 55 so that the subscriber 
can view substitute content during breaks in the 

37 

BNSDOCID: <WO 02l04O37A1_l_> 



WO 02/104037 



PCT/US02/09148 



programming content. An advertiser or broadcaster can pay 
(462) a fee or extra rates to an MSO to prevent the display 
of substitute content during a commercial break in a 
broadcast conten: or to prevent other disruptions of 
5 commercials in the broadcast content. 

Figure 7 is a flowchart diagram of a method 500 for 
substituting a substitute content in place of a live or 
pre-recorded programming content as shown on, for example, 
a television screen 444. In one embodiment, the playback 
10 engine 430 (Figure 4) may perform the method 500. The 
method 500 may run continuously or at representative 
intervals. Further, multiple instances of the method 500 
may run simultaneously. A first programming content is 
displayed (510) on, for example, the screen of the 
15 television 154 or other display device. The first 

programming content may be a live programming content, 
buffered programming content, or other programming content, 
such as a video telecast, teleconference, or prerecorded 
programming content. Next, a substitute signal is received 
20 (520) . The substitute signal may be received by the STB 
152a from, for example, a remote control device or other 
device, such as the remote control device 158. Typically, 
a viewer will send the substitute signal via the remote 
control device 158 upon viewing a commercial on the 
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television 154. However, the viewer may also send the 
substitute signal at any time, such as when live action 
takes a break between speakers in a teleconference. In 
another embodiment of the invention, the substitute signal 
5 is a tag in the broadcast, indicating the start of a 
commercial . 

In one embodiment, the substitute signal may be 
ignored if a commercial contains a signal, such as a 
particular tag, indicating that the viewing of the 
10 commercial in the programming content should not be 

disrupted. For example, this enables an advertiser, upon 
payment of an extra fee, to prevent skipping of commercials 
or substitution over commercials. 

Next, the substitute content is displayed (530) . In 
15 an embodiment of the invention, the playback engine 430 

selects the substitute content from the hard disk 162 for 
viewing based on the playback data 440. For example, if 
there is multiple substitute content on the hard disk 162, 
then the playback engine 430 can select a default 
20 programming content or pre-specif ied substitute content as 
indicated in data 440. Further, if a portion of the 
substitute content has already been viewed, as indicated by 
a tag (tags 452a & 542b in Figure 4) in the data 440, then 
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the playback engine 430 will begin displaying the 
substitute content starting from the end of the previously 
viewed content portion. In another embodiment of the 
invention, the playback engine 430 enables a viewer to 
select a substitute content to view from a list of 
substitute content in the playback data 440 upon receipt of 
a substitute signal. In another embodiment, the playback 
engine 430 selects substitute content including a live 
program that is transmitted along a specified channel. In 
another embodiment, the playback engine 4 30 selects 
substitute content including IP channel content such as 
streaming video, e-mail interaction, and so on. The IP 
channel content may be received by the STB 152a, for 
example, via the cable net 134. 

While the substitute content is being displayed (530) , 
a small window may also be optionally displayed (540) 
showing a reduced view of the first programming content, 
thus enabling a viewer to monitor the status of the first 
programming content. For example, if a viewer sends a 
substitute signal to avoid watching commercials, the viewer 
will want to return to view the first programming content 
after the end of the commercials. In one embodiment, the 
first programming content may also be continuously recorded 
by recording engine 410 while viewing the substitute 
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content. Of course, the first programming content may also 
be completely removed from the screen when the substitute 
content is being shown on the screen. 

In another embodiment, only the replayed segment of a 
5 sports broadcast are recorded by recording engine 410 or 
tagged by playback engine 4 30. To determine the replayed 
segment, various methods may be used such as detecting for 
a signal difference that characterize the replayed segment, 
or comparing the replayed segment with a prior recorded 
10 content stored in a buffer. Typically, many people are 

probably only interested in highlights (e.g., great plays 
in a baseball game) which are replayed in, for example, a 
sports news program. 

In another embodiment, the close caption of a live 
15 programming content (e.g., a sports news show) is watched 
by the viewer while the viewer is watching a pre-recorded 
or streaming content to view a highlight (e.g., a great 
play in a baseball game) . 

In another embodiment of the invention, a countdown 
20 timer may be displayed in addition to or as an alternative 
. to the smal i-.window . The countdown-timer may count down in 
30-second increments, indicating the end portion of 
successive 30-second commercials. The viewer can then 
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pause or halt the substitute content and continue viewing 
the first programming content on the screen. 

Next/ an end-substitution signal is received (550) . 
In an embodiment of the invention, the remote control 
device 158 can send the end-substitution signal to the STB 
152a. In another embodiment of the invention, a tag in the 
programming content may indicate the end of a commercial or 
the restart of the non-commercial segment in the 
programming content. Tags 4 52a and 452b may then be stored 
(560) in, for example, the playback data 440, marking the 
portion of the substitute content that has been viewed. In 
an embodiment of the invention, the playback engine 430 
stores the tags 452a and 452b. The first programming 
content is then displayed (570) in full screen on the 
television 154 by, for example, the playback engine 430. 
In addition, the substitute content may be displayed in a 
small window in reduced size and in a paused mode. 

Figure 8 illustrates a block diagram of another 
representative set top box (or other customer premise 
equipment or processing device) according to a specific 
embodiment of the invention. The STB 152b includes a 
network interface 300, processor 310, memory device 330b, 
transceiver 157, converter 320, cable modem 156, clock 350, 
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and hard disk 162 (also referred to as storage device 162) , 
all interconnected together for communication via system 
bus 340. In one embodiment, the components of STB 152b, 
except for at least some of the programming content in the 
5 memory device 330b, may be substantially similar to 

corresponding components in the STB 152a. The processor 
(controller) 310 executes instructions stored in the memory 
330b, which will be discussed in further detail in 
conjunction with Figure 10. 

10 Figure 9 is a block diagram of a memory device 330b of 

the STB 152b (Figure 8) . In one embodiment, the memory 
device 330b includes a trigger marking engine 710, marked 
triggers 720, and trigger viewing engine 730. Of course, 
other applications, such as the applications 445 in Figure 

15 4, may also be loaded in the memory 330b. Additionally, 
the engines 710, 720, and 730 may also be loaded into the 
memory 330a (Figure 4) and may enable the below described 
functionalities in the STB 152a. 

The trigger marking engine 710 enables a viewer to 
20 mark triggers (in a broadcast content) during television 
viewing for follow up viewing. In one embodiment, the 
trigger marking engine 710 stores the marked triggers in 
the marked triggers 720 where a marked trigger includes, 
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for example, a URL address (or other address information) 
associated with the marked trigger. The trigger viewing 
engine 730 enables a viewer to later select a marked 
trigger and access a URL address (or other address 
5 information) associated with the selected marked trigger. 
Thus, when, for example, a break occurs in a live 
programming content (or buffered programming content, 
recorded programming content, or other content) , a viewer 
can select and access a URL address in order to access the 
10 website (or other interactive site) that is linked to the 
URL address. Alternatively, the user may access other 
information linked to the selected marked trigger. The 
website can be shown on the television screen 44 4 in 
various formats as similarly shown, for example, by the 
15 substitute content 4 42 in Figures 5A, 5B, 5C, and/or 5D, 
while the programming content may be removed from the 
screen 444 or shown as in Figures 5B, 5C, and 5D during the 
break in the broadcast programming content. Thus, during 
the break, the viewer can view information and/or buy 
20 products or services from the displayed website (or other 
displayed information) . 

Figure 10 is a flowchart diagram of a method 800 for 
substituting live programming content or other programming 
content with trigger-associated information. In one 
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embodiment, the trigger marking engine 710 in conjunction 
•with the trigger viewing engine 730 performs the operation 
of method 800. The method 800 may run continuously or at 
representative intervals. Further, multiple instances of 
5 the method 800 may be run simultaneously on a single STB. 

First, a programming content is displayed (810) on, 
for example, the television screen 444. The programming 
content may include a live programming content or pre- 
recorded or buffered programming content. The programming 

10 content also includes triggers, which may include 

associated URL or other address information. The URL or 
other address information is simultaneously displayed on 
the television screen 444 with the programming content. If 
a viewer of the programming content is interested in the 

15 trigger, the viewer can mark the trigger by, for example, 
sending a command via a remote control device, such as 
device 158, to the STB 152b, where the command is received 
(820) . In a single viewing session, the viewer may mark a 
plurality of triggers. 

20 The marked triggers are then stored (830) . In one 

embodiment, the trigger marking engine 710 stores the 
marked triggers in marked triggers 720 which may be URL or 
other address information. Next, a skip-program signal is 
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received (840) . If there is a single stored marked 
trigger, then information associated with the single 
trigger is displayed (850) . For example, if the trigger is 
a website address, the website is loaded and displayed. In 
one embodiment, the website or information associated with 
the trigger is preloaded by the playback engine 430 so that 
when the website address is selected, the viewer during the 
conimercial break does not need to wait for the website or 
information to load. If there are multiple stored 
triggers, then information associated with the triggers may 
be cycled after a pre-defined amount of time or according 
to viewer input. Further, a viewer may be able to select 
which trigger information to display from list of stored 
triggers. In an embodiment of the invention, the trigger 
viewing engine 730 displays the information associated with 
a trigger. 

In addition to displaying information associated with 
a trigger, the programming content may also be optionally 
displayed on the television screen 444. Alternatively or 
in addition to displaying the programming content, a 
countdown timer may be displayed to indicate to the viewer 
when a commercial period ends. The timer may countdown in 
30-second increments or other increments. 
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An end-skip signal is then received (860) . In one 
embodiment of the invention, the remote control device 158 
may send the end-skip signal, which is then received by STB 
152b. Triggers, for which trigger-associated information 
5 has been displayed, may then be deleted (.870) . In one 

embodiment of the invention, the trigger viewing engine 730 
deletes displayed triggers from the marked triggers 720. 
The programming content is then displayed (880) in full 
screen mode on the screen 444. 

10 

Aggregation of Substitute Content 

Referring now to Figure 11A, shown generally at 900 is 
an interactive television system that can implement another 
specific embodiment of the invention. The system 900 

15 includes one or more head-ends 902 (or broadcast centers) 
to distribute television broadcast content and interactive 
television (sometimes referred to as x> iTV") content to a 
plurality of subscriber residences 904. The head-end 902 
includes broadcast content processing and iTV services 

20 hardware and software (including processors, servers, 
. jstor'sge media, databases , and so on) , which are show n 
generally in Figure 11A as a unit 906. The unit 906 can 
receive national broadcast content and iTV content 908 from 
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content providers, satellite feeds, servers, and/or other 
sources. The unit 906 can also receive local broadcast 
content and iTV content 910 from parties such as local 
studios, servers, content providers, and/or other sources. 

A cable modem termination system (CMTS) 912 provides 
the head-end 902 (and ultimately the subscriber residences 
904) with access to the Internet 914. A combiner 916 
places the signals and/or data provided by the unit 906 and 
the CMTS 912 into appropriate channels of a communication 
network 918. In accordance with an embodiment of the 
invention, the communication network 918 can include, for 
example, a hybrid fiber-coaxial cable distribution network 
comprising channels having a bandwidth of approximately six 
(6) MHz. Some of these channels of the communication 
network 918 are used to carry television signals (which may 
be accompanied by interactive content and other trigger 
information) , while other channels are used for upstream 
and downstream communication of data with the Internet 914. 
Some other channels (sometimes referred to as "out-of-band" 
channels) of the communication network 918 may be used to 
carry electronic program guide (EPG) information. In one 
embodiment, at least one of these out-of-band channels may 
be used to carry aggregated television programming content 
and/or related data. 
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Examples of techniques that can be used to provide 
interactive content and other trigger information from the 
unit 906 to set top boxes (or other types of customer 
premise equipment) in the subscriber residences 904 , via 
5 channels of the communication network 918 , include MPEG 
methods that multiplex special data stream (s) into the 
digital video transport stream, encoding/embedding data in 
the vertical blanking interval (VBI) portion of the analog 
television video signals (e.g., ATVEF triggers), using out- 

10 of-band modems (e.g., Digital Audio-Visual Council or 

"DAVIC" modems) to provide the interactive content, and 
other such techniques that provide information related to 
the content of the television signals. -For the channels of 
the communication network 918 that are used for 

15 communication of data to and from the Internet 914, 
protocols such as Data Over Cable Service Interface 
Specifications (DOCSIS) , transmission control 
protocol/Internet protocol (TCP/IP), hypertext transfer 
protocol (HTTP), file transfer protocol (FTP), or other 

20 suitable protocols/techniques may be used. 

In an embodiment of the invention, ATVEF triggers or 
other triggeri~ng~melih "used " to 'idehtTif y" 

specific topics of particular television programming 
content in a television signal. For instance, if the 
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sports segment of a news telecast is showing highlights of 
a particular baseball player' s homerun, that baseball 
player can be identified in the trigger. As another 
example, triggers may be used to demarcate the beginning 
and ending locations of that baseball player's highlights, 
the sports segment, the commercial breaks of the news 
telecast, or the news telecast itself. 

In an embodiment of the invention, an aggregator 940 
can aggregate user-specified television programming 
content. The aggregator 940 can include a combination of 
hardware and software (such as one or more processors, one 
or more servers, databases and/or other storage media, 
software and other machine-readable instructions stored on 
a machine-readable medium, and so on) that delivers its 
aggregated content to client terminals of subscribers via 
at least one of the channels of the communication network 
918, or via some other communication link (such as a DSL 
connection or telephone modem connection) . The aggregator 
940 can be located in the head-end 902 in one embodiment, 
while in other embodiments, the aggregator 940 can be 
located outside of the head-end 902, including somewhere in 
the Internet 914. 

The aggregator 940 can be communicatively coupled to 
the unit 906 and/or to the CMTS 912. For instance, in 
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several example implementations where the aggregator 940 
operates to provide URL addresses (or other interactive 
information peculiar to the Internet 914 or other suitable 
data communication network) to the client terminals of 

5 subscribers, the aggregator 940 can be communicatively 

coupled to the CMTS 912. It is to be appreciated that the 
aggregator 94 0 need not necessarily be coupled to the CMTS 
912 in some embodiments, provided that the aggregator 940 
can aggregate television programming content from various 

10 sources and then deliver the aggregated television 
programming content to subscribers. 

As mentioned, the aggregator 94 0 can be coupled to or 
comprise part of the unit 906, as depicted in Figure 11A. 
In such an embodiment, the aggregator 940 can have access 

15 to a database 905 that stores user-specified preferences 
for programming content as provided by a viewer. For 
instance, entries in the database 905 may indicate that the 
particular viewer wishes to aggregate all highlights for a 
particular baseball player for a specified time period. 

20 The database 905 may further identify the viewer by 

residence address, identification code of the viewer's 
"client terminal (e.g., a set top box "or recording device 
having network connectivity) , or other information to 
uniquely identify the viewer from other viewers. Then, the 

51 

BNSDOCia <WO 02 1 04037A 1 _l_> 



WO 02/104037 



PCT/US02/09148 



aggregator 940 can monitor television signals provided from 
sources 908 and 910 to the unit 906 for that baseball 
player's highlights for the specified time period, and then 
store th-e highlights in a storage unit (e.g., database 905) 
5 at the head-end 902. 

Various techniques may be used by the aggregator 940 
to locate the user-specified television programming 
content. In one embodiment, the aggregator 940 can read 
triggers of all television signals that arrive at the unit 
10 906, so as to identify the topics/content of the television 
signals as specified in the triggers, and then correlate 
the trigger content with the entries in the database 905. 
If relevant television programming content segments are 
detected from the triggers, then the aggregator 94 0 records 
15 such segments. In another embodiment, the aggregator 940 
can identify television programming content segments of 
interest by reading close-captioning data that accompanies 
the television signals. Alternatively or in addition, the 
aggregator 940 can scan EPG information that is provided to 
20 the head-end 902, with the EPG information specifying data 
such as show titles, actors, subjects, and the like. 

Various other techniques can be used by the aggregator 
940 to aggregate television programming content and to 
provide the aggregated television programming content to 
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subscribers. Examples of these techniques include 
receiving the television programming content from third- 
party entities that collect the programming content as a ■ 
service, or receiving the television programming content 
5 via direct feeds from content providers to the aggregator 
940. 

At the residence 904, a splitter 930 is coupled to 
receive the television signals (including interactive 
content and other trigger information) from the 

10 communication network 918. The splitter 930 sends the 

received television signals (including interactive content 
and other trigger information) to a first client terminal, 
such as a set top box 152, which subsequently sends the 
signals and other information to a television 154. As will 

15 be understood by those skilled in the art, the set top box 
152 includes components such as digital/analog video 
processing unit(s), VBI decoders, central processing units, 
memory, video graphics processing unit(s), software, and 
other components that can detect, extract, and process 

20 trigger information and other interactive content that is 
received from the communication network 918. For instance, 
these components can extract URL addresses from the" trigger 
information and present the URL addresses as hypertext 
links or other visual indicator on the television 154, 
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which the viewer can "click" to connect to the Internet 914 
(or other suitable data communication network) . The 
operation of these components is generally known to those 
of ordinary skill in the art. Accordingly, the specifics 
5 of their operation are not shown in Figure 11A or described 
in further detail herein for the sake of simplicity of 
explanation . 

As previously stated above, the set top box 152 can 
include or be coupled to the storage unit 162. The storage 
10 unit 162 can comprise a machine-readable storage medium 
such as a cache, buffer, memory, diskette, compact disk, 
tape, flash memory, random access memory (RAM) or the like 
and their associated hardware, in one embodiment. In an 
embodiment the storage unit 162 can comprise a hard disk 
15 such as a digital or personal video recorder (DVR or PVR) . 

In some embodiments, the splitter 930 can be coupled 
to the cable modem 156. With basic embodiments of the 
splitter 930, the cable modem 156 receives the same input 
as the set top box 152 (e.g., receives the same television 
20 signals) . With more complicated embodiments of the 

splitter 930 f the cable modem 156 and the set top box 152 
may receive different channels. Whether receiving the same 
or different channels from the splitter 930, a primary 
function of the cable modem 156 is to provide one or more 
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ancillary devices 938 with access to the Internet 914. 
Examples of the ancillary device 938 can include, but not 
be limited to, Personal Computer^ (PCs), wireless handheld 
devices, companion set top boxes, two-way advanced remote 
5 controls, or other browser-enabled display devices or 

client terminals that have the capability to communicate 
with the Internet 914. The ancillary device (s) 938 can 
include or be coupled to the storage unit 162 (or other 
storage unit) . 

10 In operation according to an embodiment, aggregated 

television programming content can be sent to the storage 
unit 162 (or to the memory 330 in the STB 152) by the 
aggregator 940. This may be done by way of broadcast, IP 
multicast, or unicast from the aggregator 940 to storage 

15 unit(s) 162 of one or more subscriber residences 904. In 
accordance with an embodiment of the invention, the 
aggregated television programming content can be sent to 
the storage unit 162 by first identifying the storage unit 
that is to receive the aggregated television programming 

20 content, such as via an identification number of the 
storage unit 162, and then sending the aggregated 
"television ^ograrraning content to the~~storage~ unit 162 in a 
format that is compatible with the particular communication 
network and with the storage unit 162 (such as a digital 
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format) . For example, the aggregated television program 
can be sent to the storage unit 162 by way of the CMTS 912, 
through a channel of the communication network 918 that 
uses the DOCSIS protocol, and then to the cable modem 156. 
5 Alternatively or in addition, an out-of-band channel may be 
used to send the aggregated television programming content 
from the aggregator 940. In yet other embodiments, the 
aggregated television programming content may be sent to 
the storage unit 162 via a different connection to the 
10 Internet 914, or the aggregated television programming 

content may be sent via one of the available channels of 
the communication network 918. 

Afterwards, incoming real-time television programming 
content (or buffered incoming programming content) is 
15 monitored by the set top box 152 for triggers that 

demarcate commercial breaks. In accordance with one 
embodiment of the invention when a trigger is received to 
indicate the beginning of a commercial break in the real- 
time television programming content, the television 154 is 
20 coupled to the storage unit 162 for playback of the stored 
programming cdntent during the commercial break. When a 
trigger is received to indicate the end of the commercial 
break, the input to the television 154 is switched from the 
storage unit 162 back to the real-time programming content. 
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During presentation of the real-time television programming 
content , a pause feature of the storage unit 162 may be 
used to pause or otherwise mark the stored television 
programming content, so that when it resumes playing during 
5 a subsequent commercial, it can begin playback at the 
proper location. 

As similarly described above, the stored programming 
content (e.g., substitute content) 442 may also be 
displayed to a viewer during a break in the incoming 

10 television program (or during a break in the buffered 

incoming television program) by having the viewer enable a 
command 426 for displaying the stored programming content. 

In accordance with another embodiment of the 
invention, the aggregated television programming content. 

15 may be delivered for presentation by way of switching to a 
channel where the programming content is carried. For 
example, the aggregator 940 may first store the aggregated 
television programming content in the storage unit 905 at 
the head-end 902. Upon recognition of a trigger that 

20 demarcates the beginning of a commercial break in a real- 
time television programming content, the set top box 152 
can send a signal" to the unit 906 at the head r end "902 to 
instruct the unit 906 to provide the aggregated television 
programming content. The unit 90 6 then sends a command 
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back to the set top box 152 that instructs the set top box 
152 to tune/switch from the channel of the real-time 
programming content (e.g., channel 945) to a different 
channel (i.e., a dedicated broadcast channel 947) where the 
aggregated programming content is to be carried. The 
aggregated television programming content is then delivered 
on that dedicated broadcast channel 947 to the set top box 
152 by the unit 906. The process can work similarly (in 
reverse) when the set top box 152 recognizes a trigger that 
demarcates the end of the commercial break, resulting in a 
switch back to the original channel 945 and the pausing of 
the aggregated television programming content. 

The set top box 152 can also tune/switch from the 
original channel 945 to the dedicated broadcast channel 947 
(and tune/switch from the dedicated broadcast channel 947 
to the original channel 94 5) by having the viewer enable 
appropriate commands 960. 

In another embodiment of the invention, aggregator 940 
directs commands 965 to STB 152 to tune to a particular 
channel (e.g., channel 947) at a particular time based on 
user-specified television programming content preferences. 
The aggregator 940 may also command STB 152 to begin 
recording and end recording so as to only record the 
programming content corresponding to the user preferences. 

58 



<WO 02104037A1_I_> 



WO 02/104037 



PCT/US02/09148 



The programming content may be recorded in, for example, 
the storage medium 162. At a later time, aggregator 940 
can then command STB 152 to substitute a live programming 
content (or buffered incoming programming content) with the 
5 recorded programming content. The advantages of recording 
locally at the STB 152 may include the following: (1) the 
programming content does not have to be captured and 
recorded at the aggregator 940 and then transmitted to the 
STB 152, and (2) dedicated bandwidth for transmission of 

10 the aggregated programming content is not required. In 
this embodiment, preferably, the STB 152 will ignore 
commands received from aggregator 94 0 if STB 152 is ' 
currently tuned to a channel, indicating that the STB is in 
use. If the STB 152 has dual channel tuning capability and 

15 dual channel recording capability, then STB 152 may display 
the programming content from a first channel and still 
record from another channel as specified by the aggregator 
940. 

In the above-embodiment, if the STB 152 has only 
20 single channel tuning and single channel recording 

capability, then programming content aggregation can be 
accomplished during periods in which the STB 152 is not 
being used (e.g., during late at night or when the user is 
on vacation) . This is because the only available resource 
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is utilized during regular television viewing. If the STB 
152 has dual channel tuning and dual channel recording 
capability, then one channel can be used for the 
aggregation of programming content, while another channel 
can be used for regular television viewing. 

It is to be appreciated that in other embodiments, the 
unit 906 can be capable of detecting triggers that 
demarcate the beginning and ending of commercial breaks. 
In such embodiments, the unit 906 can send a command 970 to 
the set top box 152 to tune to the channel where the 
aggregated television programming content is to be carried, 
instead of having the set top box 152 send a signal to the 
unit 906 that requests playback of the aggregated 
television programming content. 

In another embodiment as shown in Figure 12, various 
subscriber areas 904a to 904c can communicate with each 
other via the Internet 914 or other data communication 
network. The number of subscriber areas 904 may vary. A 
server 980 may include an application 982 that permits the 
following functions. For example, a user at the area 904a 
may be able to access directories in the memory 330b or 
hard drive 162b (in area 904b) or directories in the memory 
330c or hard drive 162c (in area 904c) to a file-sharing 
system via the Internet or network 914. The user in area 
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904a may then view and download the aggregated content in 
the accessed directories of the above-mentioned memory 
devices. The downloaded aggregated content may be stored 
in the memory devices 330a and/or 162a and displayed during 
a break in a programming content that is being received by 
the STB 1512a. 

Similarly, a user in the area 904b may be able to 
access directories of memory devices in areas 904a and 904c 
and download aggregated content that are stored in those 
memory devices. ^ 

Figure 12 is a flowchart of a method 1000 for 
collecting and displaying aggregated substitute content in 
accordance with a specific embodiment of the invention. 
The method 1000 first aggregates (1005) substitute content 
based upon user or viewer preference (s) . The aggregate 
content may then be delivered (1010) to a customer premise 
equipment (e.g., set top box 152) of the viewer. During a 
break in a currently-shown content, at least one of the 
substitute content in the aggregate content may be 
displayed (1015) . When the break in the currently-shown 
content terminates, the display of the substitute content 
may be paused or halted (1020) . The currently-shown 
content may continue to be displayed. 
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Other variations and modifications of the above- 
described embodiments and methods are possible in light of 
the foregoing teaching. For example, the components, 
modules and software shown in Figures 3 and 4 may be 
implemented in a processing device such as a personal 
computer or a suitable notebook computer. 

Further, at least some of the components of this 
invention may be implemented by using a programmed general 
purpose digital computer, by using application specific 
integrated circuits (ASICs), programmable logic devices 

(PLDs) or field programmable gate arrays (FPGAs) , or by 
using a network of interconnected components and circuits. 

Connections may be wired, wireless, by modem, and the like. 

It is also within the scope of the present invention to 

implement a program or code that can be stored in an 

electronically-readable medium to permit a computer to 

perform any of the methods described above. 

The above description of illustrated embodiments of 
the invention, including what is described in the Abstract, 
is not intended to be exhaustive or to limit the invention 
to the precise forms disclosed. While specific embodiments 
of, and examples for, the invention are described herein 
for illustrative purposes, various equivalent modifications 
are possible within the scope of the invention, as those 
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skilled in the relevant art will recognize. 

These modifications can be made to the invention in 
light of the above detailed description. The terms used in 
the following claims should not be construed to limit the 
5 invention to the specific embodiments disclosed in the 
specification and the claims. Rather, the scope of the 
invention is to be determined entirely by the following 
claims, which are to be construed in accordance with 
established doctrines of claim interpretation. 
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CLAIMS 

WHAT IS CLAIMED IS : 

1. An apparatus for performing a substitution for a 
broadcast content, the apparatus comprising: 

5 a substitute content engine configured to detect a 

break in a broadcast content and to show substitute content 
in response to the break in a broadcast content; and 

a processor configured to execute the substitute 
content engine. 

10 

2. The apparatus of claim 1 wherein the substitute 
content engine is configured to record substitute content, 

3. The apparatus of claim 1 wherein the substitute 
15 content engine is configured to display the broadcast 

content when the break terminates. 

4. The apparatus of claim 1 wherein the broadcast content 
comprises live programming content. 

20 

5. The apparatus of claim 1 wherein the broadcast content 
comprises buffered programming content. 

6. The apparatus of claim 1 wherein the broadcast content 
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comprises video telecast content. 

7. The apparatus of claim 1 wherein the broadcast content 
comprises teleconference content. 

5 

8. The apparatus of claim 7 wherein the teleconference 
content comprises remote learning session broadcast 
content. 

10 9. The apparatus of claim 7 wherein the teleconference 
content comprises video conference content. 

10. The apparatus of claim 1 wherein the break in the 
broadcast content includes a commercial segment. 

15 

11. The apparatus of claim 1 wherein the break in the 
broadcast content includes a commercial skip segment. 

12. The apparatus of claim 1 wherein the break in the 
20 broadcast content represents a broadcast stoppage. 

13. ~ The apparatus of claim 1 wherein the break in the 
broadcast content represents non-activity in a broadcast 
event . 
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14. The apparatus of claim 1 wherein the substitute 
content comprises live broadcast content. 

5 15. The apparatus of claim 1 wherein the substitute 
content comprises buffered broadcast content. 

16. The apparatus of claim 1 wherein the substitute 
content comprises recorded broadcast content. 

10 

17. The apparatus of claim 1 wherein the substitute 
content comprises Internet Protocol channel content. 

18. The apparatus of claim 17 wherein the Internet 
15 Protocol channel content comprises streaming video. 

19. The apparatus of claim 17 wherein the Internet 
Protocol channel content comprises electronic mail 
interaction . 

20 

20. The apparatus of claim 1 wherein the substitute 
content is displayed in full screen mode. 

21. The apparatus of claim 1 wherein the substitute 
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content is displayed as a main screen view and the 
broadcast content is displayed as a picture-in-picture - 

22. The apparatus of claim 1 wherein the substitute 
5 content is displayed in the screen foreground and the 

broadcast content is displayed in the screen background . 

23. The apparatus of claim 1 wherein the substitute 
content is displayed in a first screen area and the 

10 broadcast content is displayed in a second screen area. 

24. The apparatus of claim 1 wherein the substitute 

. content is displayed in response to a user-enabled command. 

15 25. The apparatus of claim 1 wherein the substitute 

content is automatically displayed upon detection of the 
break in the broadcast content. 

26. The apparatus of claim 1 wherein the viewing engine is 
20 capable to display a timer to indicate a time length for 
showing the substitute content. 



27. The apparatus of claim 1 wherein the substitute 
content is automatically displayed in response to the break 
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in the broadcast content. 

28. The apparatus of claim 1 wherein the viewing engine is 
capable to pause the substitute content when the break in 

5 the broadcast content terminates. 

29. The apparatus of claim 1 wherein the substitute 
content engine is configured to record the broadcast 
content when the substitute content is displayed. 

10 

30. The apparatus of claim 1 wherein the substitute 
content engine is configured to pause the broadcast content 
when the break ends. 

15 31. The apparatus of claim 1 wherein the broadcast content 
includes a tag to prevent disruption of the display of the 
broadcast content during the occurrence of the break. 

32. The apparatus of claim 31 wherein a first party pays 
20 an operator to include the tag in the broadcast content. 

33. The apparatus of claim 31 wherein a subscriber pays an 
operator to disable the tag. 
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34. The apparatus of claim 1 wherein the broadcast ' content 
includes a trigger. 

35. The apparatus of claim 1, wherein the substitute 
5 content is associated with the trigger. 

36. The apparatus of claim 35 wherein the trigger is 
associated with an information site. 

10 37. The apparatus of claim 36 wherein the information site 
is loaded prior to a break in the broadcast content. 

38. The apparatus of claim 1, further comprising: 

an aggregator communicatively coupled to the customer 
15 premise equipment and capable to aggregate at least one 
substitute content based upon a viewer preference. 

39. The apparatus of claim 1 wherein the broadcast content 
is recorded to permit viewing of a missed segment in the 

20 broadcast content when the display of the substitute 
content terminates . 

40. The apparatus of claim 1 wherein a pre-selected 
segment of the broadcast content is recorded during display 
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of the substitute content. 

41. The apparatus of claim 4 0 wherein the pre-selectecl 
segment is content replayed during the showing of the 

5 broadcast content. 

42. The apparatus of claim 40 wherein the pre-segment is 
viewed while displaying close caption text from the 
broadcast content . 

10 

43. An apparatus for substituting broadcast content 
displayed on a screen, the apparatus comprising: 

a customer premise equipment capable to receive 
broadcast content ; 
15 the customer premise equipment including: 

a marking engine capable to mark at least one trigger 
in a broadcast content; and 

a viewing engine communicatively coupled to the 
marking engine and capable to permit the display of 
20 information associated with a selected marked trigger on 
the screen. 

44. The apparatus of claim 4 3 wherein the displayed 
information is an interactive site. 
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45. The apparatus of claim 44 wherein the interactive site 
can receive a purchase or request from the viewer. 

5 46. An apparatus for substituting broadcast content 
displayed on a screen, the apparatus comprising: 

means for determining an occurrence of a break in a 
broadcast content; and 

coupled to the determining means, means for displaying 
10 substitute content on the screen in response to the 
occurrence of the break. 

47. An apparatus for substituting broadcast content 
displayed on a screen, the apparatus comprising: 

15 means for marking at least one trigger in a broadcast 

content; 

coupled to the marking means, means for determining an 
occurrence of a break in the broadcast content; and 

coupled to the marking means, means for displaying 
20 information associated with a selected marked trigger on 
the screen in response to the occurrence of the break. 

48. A system for preventing the disruption of break 
segments in a broadcast content, the system comprising: 
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a first stage configured to insert a trigger in a 
broadcast content, the trigger configured to prevent the 
disruption of a break in the broadcast content; 

a transmission path configured to transmit the trigger 
5 from the first stage; and 

a destination stage configured to receive the trigger 
along the transmission path and to prevent the disruption 
of the break in the broadcast content. 

10 49. The system of claim 48 wherein the trigger is inserted 
in the broadcast content based upon payment of a fee. 

50. A system for permitting the display of content during 
break segments in a broadcast content, the system 
15 comprising : 

a first stage configured to insert a trigger in a 
broadcast content, the trigger configured to prevent the 
disruption of a break in the broadcast content; 

a transmission path configured to transmit the trigger 
20 from the first stage; and 

a destination stage configured to receive the trigger 
along the transmission path and to disable the trigger to 
permit display of substitute content during the break in 
the broadcast content. 
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51. The system of claim 50 wherein the trigger can be 
disabled based upon payment of a fee. 

5 52. An apparatus for substituting broadcast content 
displayed on a screen, the apparatus comprising: 

a customer premise equipment capable to receive 
broadcast content; 

the customer premise equipment including a viewing 
10 engine capable to permit the display of substitute content 
in response to a break in the broadcast content. 

53. The apparatus of claim 52 wherein the broadcast 
content is displayed when the break terminates. 

15 

54. The apparatus of claim 52 wherein the broadcast 
content comprises live programming content. 

55. The apparatus of claim 52 wherein the broadcast 
20 content comprises buffered programming content. 

56. The apparatus of claim 52 wherein the" broadcast 
content comprises video telecast content. 
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57 . The apparatus of claim 52 wherein the broadcast 
content comprises teleconference content. 

58. The apparatus of claim 57 wherein the teleconference 
content comprises remote learning session broadcast 
content. 

59. The apparatus of claim 57 wherein the teleconference 
content comprises video conference content. 

60 . The apparatus of claim 52 wherein the break in the 
broadcast content includes a commercial segment. 

61. The apparatus of claim 52 wherein the break in the 
broadcast content includes a commercial skip segment. 

62. The apparatus of claim 52 wherein the break in the 
broadcast content represents a broadcast stoppage. 

63. The apparatus of claim 52 wherein the break in the 
broadcast content represents non-activity in a broadcast 
event. 

64 . The apparatus of claim 52 wherein the substitute 
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content comprises live broadcast content. 



65. The apparatus of claim 52 wherein the substitute 
content comprises buffered broadcast content. 

5 

66. The apparatus of claim 52 wherein the substitute 
content comprises recorded broadcast content. 

67. The apparatus of claim 52 wherein the substitute 
10 content comprises Internet Protocol channel content. 

68. The apparatus of claim 67 wherein the Internet 
Protocol channel content comprises streaming video. 

15 69. The apparatus of claim 67 wherein the Internet 
Protocol channel content comprises electronic mail 
interaction. 

70. The apparatus of claim 52 wherein the substitute 
20 content is displayed in full screen mode. 

"71." The apparatus of claim 52 wherein the substitute 
content is displayed as a main screen view and the 
broadcast content is displayed as a picture-in-picture . 
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72. The apparatus of claim 52 wherein the substitute 
content is displayed in the screen foreground and the 
broadcast content is displayed in the screen background. 

73. The apparatus of claim 52 wherein the substitute 
content is displayed in a first screen area and the 
broadcast content is displayed in a second screen area. 

74. The apparatus of claim 52 wherein the substitute 
content is displayed in response to a user-enabled command. 

75. The apparatus of claim 52 wherein t-he substitute 
content is automatically displayed upon detection of the 
break in the broadcast content. 

76. The apparatus of claim 52 wherein the viewing engine 
is capable to display a timer to indicate a time length for 
showing the substitute content. 
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SYSTEM & APPARATUS FOR DISPLAYING SUBSTITUTE CONTENT 



CROSS-REFERENCE TO RELATED APPLICATION 

The present application is a continuation-in-part of 
10 and claims the benefit of U.S. Patent Application Serial 
No. 09/883,098, entitled "ADVERTISEMENT SWAPPING USING AN 
AGGREGATOR FOR AN INTERACTIVE TELEVISION SYSTEM" by Douglas 
A. Miller, filed June 14, 2001. Application 09/883,098 is 
fully incorporated herein by reference. 

15 

TECHNICAL FIELD 

This disclosure relates generally to television 
programming content substitution methods and systems, and 
more particularly but not exclusively, to techniques for 
20 substituting programming content during commercial breaks 
or other breaks in a broadcast content. 



" BACKGROUND" " " — " 

Conventionally, personal video recorders (PVRs) enable 
25 a viewer to record television programming content and then 
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to view the recorded television programming content at a 
later time. Further, PVRs may enable a viewer to record 
specified programming content on a regular basis and to 
delete the recorded programming content after a pre- 
5 specified amount of time. For example, a viewer may 

program a PVR to record a daily news show (such as the 5 
o'clock news) on a daily basis and then to automatically 
delete the recorded news show after one day since the news 
content would be stale. 

10 In order to view the recorded programming content, a 

viewer typically proceeds through multiple steps, which may 
include loading a user interface <UI) for display, 
selecting a "view pre-recorded programming content" option 
from a menu in the UI, and then selecting a pre-recorded 

15 programming content from a list of several pre-recorded 
programming content. 

PVRs also enable a viewer to pause "live" programming 
content (and/or buffered programming content) and to skip 
commercials and/or other portions of live television 
20 programming content (or buffered programming content) . For 
example, a viewer can skip commercials when viewing 
programming content on a television coupled to a PVR by 
pressing a "skip" button of the PVR remote control device. 

2 
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Thus, advertisers are concerned that the ability of 
viewers to skip commercials when viewing programming 
content with PVRs will lead to losses in revenue. 
Additionally, a viewer, while watching live TV may channel 
5 surf and lose the buffered portion of the content shown in 
the original channel. 

Accordingly, there is need for a new programming 
content substitution system and method that will overcome 
the deficiencies mentioned above. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Non-limiting and non-exhaustive embodiments of the 
present invention are described with reference to the 
following Figures, wherein like reference numerals refer to 
5 like parts throughout the various views unless otherwise 
specified. 

Figure 1 is a block diagram of an interactive video 
casting system that can implement an embodiment of the 
invention . 

10 Figure 2 is a block diagram of another interactive 

video casting system that can implement an embodiment of 
the invention. 

Figure 3 is a block diagram of a customer premise 
equipment, such as a set top box, according to an 
15 embodiment of the invention. 

Figure 4 is a block diagram of a memory device of the 
customer premise equipment of Figure 3. 

Figure 5A is a block diagram of a television screen 
where the substitute content is shown as a full screen view 
20 across the television screen. 

Figure 5B is a block diagram of a television screen 
where the substitute content is shown as the main screen 
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/ 

view, while the programming content is shown as a picture- 
in-picture . 

Figure 5C is a block diagram of a television screen 
where the substitute content is shown on the television 
5 screen foreground, while the programming content is shown 
on the television screen background. 

Figure 5D is a block diagram of a television screen 
where the substitute content is shown on one portion of a 
split screen view, and the programming content is shown on 
10 another portion of the split screen view. 

Figure 6A is a block diagram of a programming content 
that includes a disable feature to prevent disruption of 
the viewing of commercial segments in the programming 
content . 

15 Figure 6B is a flowchart of a method of preventing the 

viewing of substitute content during program breaks. 

Figure 6C is a flowchart of a method of permitting a 
user to view substitute content during program breaks. 

Figure 7 is a flowchart of a method for substituting a 
20 live or pre-recorded (buffered) programming content with 

substitute content, in accordance with an embodiment of the 
invention . 
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Figure 8 is a block diagram of another representative 
customer premise equipment, such as a set top box, 
according to an embodiment of the invention; 

Figure 9 is a block diagram of a memory device of the 
customer premise equipment of Figure 8. 

Figure 10 is a flowchart of a method for substituting 
a live or other programming content with trigger-associated 
information, in accordance with an embodiment of the 
invention. 

Figure 11A is a block diagram of another interactive 
television system that can implement an embodiment of the 
invention. 

Figure 11B is a block diagram of a system for 
permitting sharing of aggregated substitute content. 

Figure 12 is a flowchart of a method of collecting and 
displaying aggregated substitute content, in accordance 
with an embodiment of the invention. 
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DETAILED DESCRIPTION OF THE ILLUSTRATED EMBODIMENTS 

Embodiments of a system and method for substituting a 
live or pre-recorded (or buffered) programming content 
(i.e., broadcast content) with a substitute content are 
5 disclosed herein. The substitute content may be, for 

example, a viewer-selected content or default content. As 
an overview, in an embodiment, a broadcast content 
substitution system includes a customer premise equipment 
(or other processor) that can permit the display of 
10 broadcast content. In response to the occurrence of a 
break in the broadcast content, the customer premise 
equipment can permit the display of substitute content. 
The break may be, for example, a commercial in a live 
broadcast content, a commercial skip segment in a buffered 
15 live broadcast content, a broadcast stoppage, or a non- 
activity in a broadcast event. When the break in the 
broadcast content terminates, the customer premise 
equipment can again permit the display of the broadcast 
content. During the occurrence of the break, the customer 
20 premise equipment may permit the display of the substitute 
content and images related to the break in the broadcast 
content . 

In another embodiment, the invention broadly provides 
a customer premise equipment (or other processing device) 
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that includes a recording engine and a playback engine. 
The recording engine enables a viewer to select programming 
content to record on, for example, a regular basis, which 
is sometimes referred to as a "season's pass." For 
5 example, a viewer can program the recording engine to 
record a 5 o'clock news show on a daily basis. The 
recording engine may also automatically delete recorded 
programming content after a selected amount of time. 

The playback engine, upon receiving a user-enabled 

10 command signal, can substitute the substitute content in 
place of the currently-displayed live programming content 
or buffered programming content. In another embodiment, 
commercial breaks in the currently-displayed live 
programming content or buffered programming content are 

15 automatically detected, and the substitute content is 

automatically displayed during the commercial breaks. The 
substitute content can be a previously-recorded programming 
content, a live programming content, or content from an 
Internet Protocol (IP) channel, such as streaming video, 

20 electronic mail interaction, and so on. The electronic 
mail interaction would be automatically activated during, 
for example, commercial breaks and de-activated (or 
paused/halted) during the non-commercial segments in the 
live programming content or buffered programming content. 
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The substitute content may be specified by the user or may 
be a default content. 

In another embodiment, the substitute content is 
displayed during breaks in a video telecast or 
5 teleconference (where a teleconference may be, for example, 
a Remote Learning Session or Video Conference) . A break in 
a live action in a video telecast or teleconference may 
occur, for example, when there is a change in speakers, 
during the time when equipment is -set up, during unforeseen 
10 delays in the presentation, and so on. 

The substitute content may, for example, be displayed 
in full screen mode on a television. Alternatively or in 
addition, the substitute content may be displayed as the 
main screen view, while the live or other programming 
15 content is displayed as a picture-in-picture on the 
television screen. Alternatively or in addition, the 
substitute content may be displayed as a picture overlay on 
the television screen foreground, while the live or other 
programming content is displayed on the television screen 
20 background. Alternatively or in addition, the substitute 
content and the live or other programming content may be 
displayed in separate screen areas as split screen views on 
the television screen. 



9 
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In another embodiment, the invention broadly provides 
a customer premise equipment (or other processing device) 
that includes a trigger marking engine and a trigger 
viewing engine. The customer premise equipment can receive 

5 programming content that includes Advanced Television 
Enhancement Forum (ATVEF) triggers or other types of 
triggers. One type of trigger, ATVEF B, may also be used 
and includes content as well as triggers. If a viewer is 
interested in a particular trigger that appears on the 

10 television screen, then the viewer can, for example, enable 
a command to signal the customer premise equipment to mark 
a Uniform Resource Locator (URL) address (or other address 
information peculiar to the Internet or other data 
communication network) where the address is associated with 

15 the trigger of interest. The URL address or other address 
is then stored for later viewing and access. During a 
commercial or other breaks in a live or buffered, broadcast 
content, the viewer can then signal the customer premise 
equipment to show and access the stored URL address (or 

20 other address) in order to view information associated with 
previous triggers of interest. When a website (or other 
inf ormation) associated with the accessed URL address (or 
other address) is shown, the website may, for example, be 
shown in full screen on the television. Alternatively or 
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in addition, the website may be displayed on the television 
screen, while the live or buffered broadcast content is 
displayed as a picture-in-picture on the television screen. 
Alternatively or in addition, the website may be . displayed 
5 as a picture overlay on the television screen foreground, 
while the live or buffered broadcast content is displayed 
on the television screen background. Alternatively or in 
addition, the website and the live or buffered broadcast 
content may be displayed in separate areas as split screen 

10 views on the television screen. The viewer can then, for 
example, obtain information or buy items or services in the 
displayed website. The viewer may then return to viewing 
the live or buffered broadcast content by enabling an 
appropriate command, for example, via the remote control 

15 device . 

In another embodiment of the invention, an aggregator 
can be used to aggregate user-selected content . The 
aggregated user-selected content can then be provided for 
playback to the user, for example, during commercial breaks 
20 in a live or buffered programming content or during breaks 
in a video telecast or teleconference. For example, the 
aggregator can be used to locate and collect programming 
content (or portions of a programming content) related to a 
specific topic as specified by the user. The collected 



11 



BNSDOCID: <WO 02104037A1_IA> 



WO 02/104037 



PCT/US02/09148 



programming content can then be displayed during commercial 
breaks in a live or buffered broadcast or during breaks in 
a video telecast or teleconference. In one embodiment, the 
aggregator can provide the aggregated user-selected content 
to a storage unit (such as a PVR) for subsequent playback. 
In another embodiment, a customer premise equipment (such 
as a set top box) can perform channel switching to switch 
from a channel where the live or buffered programming 
content is showing to another channel (e.g., a dedicated 
broadcast channel) where the aggregated user-selected 
content can be shown. 

In the description herein, numerous specific details 
are provided, such as examples of system components and/or 
methods, to provide a thorough understanding of embodiments 
of the invention. One skilled in the relevant art will 
recognize, however, that the invention can be practiced 
without one or more of the specific details, or with other 
systems, methods, components, materials, parts, and/or the 
like. In other instances, well-known structures, 
materials, or operations are not shown or described in 
detail to avoid obscuring aspects of the invention. 

Reference throughout this specification to "one 
embodiment ", "an embodiment", or "a specific embodiment" 
means that a particular feature, structure, or 
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characteristic described in connection with the embodiment 
is included in at least one embodiment of the present 
invention. Thus, the appearances of the phrases "in one 
embodiment", "in an embodiment", or "in a specific 
embodiment" in various places throughout this specification 
are not necessarily all referring to the same embodiment. 
Furthermore, the particular features, structures, or 
characteristics may be combined in any suitable manner in 
one or more embodiments. 

Additionally, the signal arrows in the 
drawings/Figures are considered as exemplary and are not 
limiting, unless otherwise specifically noted. 
Furthermore, the term "or" as used in this disclosure is 
generally intended to mean "and/or" unless otherwise 
indicated. Combinations of components or steps will also 
be considered as being noted, where terminology is foreseen 
as rendering the ability to separate or combine is unclear. 

It will also be appreciated that one or more of the 
elements depicted in the drawings/figures can also be 
implemented in a more separated or integrated manner, or 
even removed or rendered as inoperable in certain cases, as 
is useful in accordance with a particular application. 

Figure 1 is a block diagram of an interactive video 
casting system 100 that can implement a specific embodiment 
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of the invention. Of course, the illustrated system 100 is 
provided for example purposes only, and other systems or 
topologies may be used within the scope of the invention. 
In an embodiment, a production company 104 produces 
5 programming content for transmission to viewers. The 
transmission is sent via a satellite transmission 
transceiver 112 over an uplink channel to a satellite 102. 
The satellite 102 then transmits the programming content 
over a downlink channel to a local studio 106. The- local 

10 studio 106 can insert additional programming content (e.g., 
regional programming content) and/or advertisements as 
needed into the programming content. The programming 
content with the insertions is then transmitted from the 
local studio 106 to a cable service provider 108. The 

15 television programming content may be downloaded to a 

receiving station, such as a broadcast center (e.g., cable 
distribution head-end) (not shown) of the cable service 
provider 108, rather than or in addition to the local 
studio 106. A reverse channel from the cable service 

20 provider 108 to the local studio 106 may be provided so 

that the local studio 106 can insert additional programming 
content and feed the television signal back to the cable 
service provider 108. The cable service provider 108 then 
delivers the television signal over a cable network 134 to 
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cable subscribers. 

The cable network 134 is provided by the cable service 
provider 108 to distribute the programming content to cable 
subscribers. A set top box (STB) 152, typically located on 
5 the premises of a cable television subscriber, receives the 
programming content or television signal from the cable net 
134, and delivers the television signal to the subscriber's 
television set (or other display device) 154. In some 
specific embodiments, alternatively or in addition, the 

10 television signal can be broadcast over a wireless medium 
and received by a traditional aerial antenna or by a 
satellite dish, and then delivered to the set top box 152. 
Alternatively or additionally, features and functionality 
of the set top box 152 may be integrated into a type of 

15 advanced television or other display device. 

Moreover, embodiments of the invention can use other 
types of broadcast media, including but not limited to, 
digital cable systems, satellite, very-high-data-rate 
digital subscriber line (VDSL), web casts, and the like. 
20 The features provided by the television set 154 can also be 
"provisioned, in one " embodiment , by a .personal computer (PC) 
suitably configured with an adapter to convert television 
signals into a digitized format, and then to deliver the 
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television signals to the video portion of the computer for 
display. It is "noted that the invention is not limited to 
any one configuration of display hardware as embodiments of 
the invention can be realized using alternative reception 
5 and display arrangements, as known to those skilled in the 
art. 

In accordance with an embodiment of the invention, a 
connection to a communication network is provided for the 
cable subscriber. In one embodiment, the connection can be 

10 made via a cable modem 156 over a bi-directional 

communication link 155 to a cable modem termination system 
within the cable provider's 108 equipment. The connection 
continues to a data communication network, such as the 
Internet, by way of a public switched telephone network 

15 (PSTN) 132.. The PSTN network 132 is provided herein as an 
example, and it is understood that other types of networks 
may be used for connectivity to the Internet. A cable 
modem arrangement can be used because of its high bandwidth 
capability. In situations where some cable companies are 

20 not equipped to provide cable modem service to their 

customers, various other arrangements can be made. For 
example, a conventional modem connection can be used to 
access the Internet over a telephone line. As another 
example, Internet access can be gained over a DSL 
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connection or an integrated services digital network (ISDN) 
connection using a telephone line. Wireless systems are 
also available for providing Internet access. In one 
embodiment, downstream data transmission may occur via 
5 cable or satellite, and upstream data transmission may 
occur via a telephone line. 

It is noted that the Internet is chosen as an example 
of a data communication network because it is a well- 
established network, and connectivity to the Internet is 
10 easily made. However, it is noted that a global 

communication network, such as the Internet, is not 
required to practice other embodiments of the invention. A 
locally provided and maintained communication network may 
be used in an embodiment of the invention. 

15 Continuing with Figure 1, the set top box 152 can 

include a transceiver 157, such as an infrared (IR) or 
radio frequency (RF) transceiver, that can exchange signals 
with a remote control unit 158 or other user input device. 
The set top box 152 can be a component that is separate 

20 from the television set 154 as shown in Figure 1, or its 
features can be built into circuitry of the . television set 
154 (e.g., an interactive television set). The set top box 
152 enables a viewer to select television programming 
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content to view and then delivers the television 
programming content to the television set 154. A storage 
unit 162 can also be coupled to or be a part of the set top 
box 152. The storage unit 162 can include a machine- 
5 readable storage medium such as a cache, buffer, memory, 
diskette, compact disk, tape, or the like and their 
associated hardware, in one embodiment. In another 
embodiment, the storage unit 162 can include a video 
cassette recorder (VCR) . In another embodiment, the 
10 storage unit can include a hard disk such as a digital or 
personal video recorder (DVR or PVR) . 

As noted above, the local studio 106 can insert 
additional programming content into the received 
transmission, for example, to provide cable content that 

15 includes locally provided channels. The programming 

content is then distributed to customers over the cable 
network 134. In addition to local programming content 
insertion, the local studio 106 can insert advertising 
content. Product supplemental information relating to the 

20 advertising for participating merchants 122 can also be 
inserted. Product supplemental information can include 
information relating to the goods or services being 
advertised in the commercial. In addition to goods and 
services, coupons and other information services can be 
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made available to the viewer, which in one embodiment can 
be obtained via the merchant's 122 web site 124 on the 
Internet. Triggers, such as Advanced Television 
Enhancement Forum (ATVEF) triggers, which are related to 
5 the web site 124 and/or to its contents, can be 

continuously updated as the television broadcast is being 
received. 

As noted above, the triggers, resources, or 
announcements can be inserted by the originating 
10 broadcaster 104, a local broadcaster 106, or by the cable 
system operator 108. 

Figure 2 is another block diagram showing an example 
of an interactive video casting system 200 and illustrates 
another point of insertion of the product supplemental 
15 information. In a specific embodiment shown in Figure 2, a 
merchant 222, operating a web site 224, is located such 
that the merchant 222 is capable of inserting commercial 
prior to the uplink transmission to the satellite 102. 

As noted above, Internet access is not necessary to 
20 practice the invention. A locally provided network may, 
for example, be used to practice an embodiment of the~ 
invention. The cable provider 108 can supply the foregoing 
features, for example, by providing a web site or "walled 

19 



BNSDOCID: <WO 02104037A1 JA> 



WO 02/104037 



PCT/US02/09148 



garden" that is accessed by its subscribers. In such a 
case, the cable provider 108 serves as an intermediary and 
submits the purchases to the actual merchants 122 or 222. 

In another embodiment, programming content and/or 
5 other data may be distributed to a subscriber/customer via 
a satellite TV delivery system, which may include a direct 
broadcast satellite (DBS) system. A DBS system may 
include, for example, a small 18-inch satellite dish (which 
is an antenna that can receive a satellite broadcast 
10 signal); a digital integrated receiver/decoder (IRD), which 
separates each channel, and decompresses and translates the 
digital signal so that a television can show the digital 
signal; and a remote control that transmits command 
signals. Programming for a DBS system may be distributed 
15 by, for example, multiple high-powered satellites in geo- 
synchronous orbit, with each satellite having multiple 
transponders. Compression (e.g., Moving Pictures Experts 
Group or MPEG based compression) may be used to increase 
the amount of programming content that can be transmitted 
20 in the available bandwidth. 

A digital broadcast center may be used to gather 
programming content, ensure the digital quality of the 
programming content, and transmit the signal up to the 
satellites for delivery. Programming content may be 
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provided to the broadcast center from content providers 
(e.g., ABC, CNN, ESPN, and so on) via satellite, fiber 
optic cable, and/or special digital tape. Satellite- 
delivered programming content are typically immediately 
5 digitized, encrypted, and up-linked to the orbiting 

satellites. The satellites re-transmit the signal back 
down to every earth-station (or, in other words, every 
compatible DBS system receiver dish at, for example, homes 
and businesses of customers) . 

10 Some programming content may be recorded on digital 

videotape in the broadcast center, and the programming 
content can be broadcast at later time or day. Before any 
recorded programming content are viewed by customers, 
technicians may use post-production equipment to view and 

15 analyze each tape to, for example, ensure audio and video 
quality. Tapes may then be loaded into, for example, a 
robotic tape handling system, and playback may be triggered 
by a computerized signal sent from a broadcast automation 
system. Back-up videotape playback equipment may ensure 

20 uninterrupted transmission at all times. 

Figure 3 illustrates a block diagram of a 
representative set top box (or other suitable customer 
premise equipment or processing device) 152a according to a 
specific embodiment of the invention. It is noted that the 
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STB 152a described below is one of the possible embodiments 
of the STB 152 that is shown in Figure 1 and Figure 2. It 
is further noted that the components shown in Figure 3 may 
be implemented in other types of customer premise equipment 
5 or other types of processing devices. In an embodiment, 
the STB 152a includes a network interface 300, processor 
310, memory device 330a, transceiver 157, converter 320, 
cable modem 156, optional clock 350, and hard disk 162 
(which may be any suitable storage device), all 

10 interconnected together for communication via a system bus 
34 0. For purposes of explaining the functionality of the 
invention, other known components or modules or software in 
a set top box have been omitted in Figure 3. Network 
interface 300 connects the STB 152a to the cable network 

15 134 (Figure 1). In an alternative embodiment, the cable 
modem 156 or the converter 320 may provide some or all of 
the functionality of the network interface 300, and thus, 
the network interface 300 may be omitted. In another 
embodiment, the network interface 300 may also provide some 

20 or all of the functionalities of the converter 320 and 

cable modem 156, and as a result, the converter 320 and/or 
cable modem 156 may be omitted in this particular 
embodiment. Thus, the channels shown in Figure 3 (e.g., 
channel 322) may be coupled to the cable modem 156 and/or 
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to the network interface 300. 

The processor (controller) 310 executes instructions 
stored in memory 330a, which will be discussed in further 
detail in conjunction with Figure 4. The processor 310 
also typically performs other management functions of the 
STB 152a. 

The transceiver 157 can include an infrared (IR) or 
radio frequency (RF) transceiver (or other transceiver 
types) that can exchange signals with the remote control 
unit 158 (Figure 1 or Figure 2) or other user input device. 
The converter 320 can convert, if necessary, digitally 
encoded broadcasts to a format usable by the television 154 
(Figure 1 or Figure 2) . In addition, the converter 320 can 
convert other data received in an out-of-band portion of a 
broadcast, such as television scheduling information to a 
format that can be stored in the memory 330a or hard disk 
162. 

The cable modem 156 can transmit and receive digital 
information, such as television scheduling information, if 
not included in the out-of-band portion of a broadcast. In 
alternative embodiments, the cable modem 156 may be a 
conventional modem for use over telephone lines or may 
include any other component or module for transmitting and 
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receiving digital data. 

The STB 152a may include the hard disk 162 or, in an 
alternative embodiment , the hard disk 162 may be external 
to STB 152a. For example, in one embodiment, a PVR (not 
5 shown) may include the hard disk 162 and may be coupled to 
the STB 152a. The hard disk 162 may store television 
programming content in MPEG format or other format or 
combination of formats. 

The optional clock 350 is typically a digital clock 
10 for use with the engines that can be stored in the memory 
device 330a (see Figure 4) . The clock 350 can be updated 
regularly by using the cable modem 156 to contact an online 
clock, such as the Atomic Clock at the U.S. Naval 
Observatory, or to contact a head-end at the provider 108 
15 so that the clock 350 is synchronized with the clock of 
provider 108. Alternatively, the clock 350 may be set 
manually by a viewer or may be updated through other 
methods known to those skilled in the art. 

Figure 4 is a block diagram of a memory device 330a in 
20 the STB 152a (Figure 3) . In the specific embodiment shown 
in Figure 4, the memory device 330a includes a recording 
engine 410; recording data 420; playback engine (or viewing 
engine) 430; playback data 440; and applications 445. The 
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various engines discussed herein may be, for example, 
software, commands, data files, programs, code, modules, 
instructions, or the like. It is also understood that the 
various engines described herein may be combined. For 
5 example, the recording engine 410 and playback engine 430 
may be combined as a substitute content engine 4 32. 
Furthermore, the various engines (e.g., applications 445) 
described herein may reside in other storage devices such 
as hard disk 162. 

10 The recording engine 410 may enable a viewer to 

schedule a ''season's pass" recording schedule, i.e., to 
record specific programming content on a daily or other 
regularly scheduled basis. The recording engine 410 may 
also enable a viewer to schedule the recording of a 

15 particular programming content at a specific day/time. The 
programming content may be received by the STB 152a via, 
for example, the cable net 134. The recording engine 410 
then can record the programming content onto the hard disk 
162 in MPEG format or other format for later viewing on the 

20 television 154. 

The recording data 420 includes viewer-specified 
recording instructions for the recording engine 410, such 
as the particular channel (s) to record, the time/day to 
start recording, and the length of the recording time 
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and/or the time to stop recording. The playback engine 430 
sends the recorded programming content to the converter 320 
to convert to a format capable to be displayed on the 
television 154, if necessary. The playback engine 430 may 
5 also convert the programming content into a format 
displayable by the television 154 depending on the 
capabilities of the converter 320. 

In one embodiment, upon receiving a user-enabled 
command signal 426 (e.g., from the user-controlled remote 

10 control device 158), the playback engine 430 can set the 
substitute content 442 (see Figure 5A) in place of the 
currently displayed live programming content or buffered 
programming content as shown on the screen 44 4 of the 
television 154. Of course, the user can enable the command 

15 signal 42 6 by directly actuating a button or switch on the 
set top box 152a, instead of using, for example, the remote 
control device 158. In another embodiment, commercial 
breaks in the currently displayed live or buffered 
programming content are automatically detected by the 

20 playback engine 430, and the substitute content 442 is 

automatically displayed during the commercial breaks and 
paused (or removed from the screen 44 4) when the commercial 
break ends and the main program in the broadcast resumes. 
Commercial breaks in the live or buffered programming 

26 

BNSDOCIO: <WO 02104037A1_IA> 



WO 02/104037 



PCT/US02/09148 



content, can be detected by having the set top box 152a 
monitor for triggers or tags that demarcate commercial 
breaks . 

More than one substitute content 442 may be displayed 
5 by the playback engine 430. The substitute content 442 can 
be a previously-recorded programming content (e.g.', 
advertisements, news, movies, and/or other types of shows) 
as recorded by the recording engine 410, live programming 
content as selected on a particular channel by the playback 
10 engine 430, or content from an Internet Protocol channel, 
such as streaming video, electronic mail interaction, and 
so on. The electronic mail interaction would be 
automatically activated during, for example, commercial 
breaks and de-activated (or halted) during the non- 
15 commercial segments in the live or buffered programming 
content. The substitute content 442 may be, for example, 
stored content as specified by the user, may come from a 
selected channel as specified by the user, may be content 
from a default channel, and/or may be stored content that 
20 is chosen by default by the playback engine 430. As 
described below with reference to Figure 11A, the 
substitute content 4 42 may also be content from an 
aggregated content collection that is based upon a criteria 
selected by the viewer. 
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In another embodiment, the substitute content 442 is 
displayed for a particular amount of time (e.g., 30 
seconds) by the playback engine 430 after a commercial 
break occurs in the programming content 44 6 currently 
5 displayed on the television screen 444 (see, e.g., Figure 
5C) . 

In another embodiment, the substitute content 442 is 
displayed during breaks in a video telecast or 
teleconference (where a teleconference may be, for example, 
10 a Remote Learning Session or Video Conference) that are 
shown in the television screen 444. A break in a live 
action in a video telecast or teleconference may occur, for 
example, when there is a change in speakers, during the 
time when equipment is set up, during unforeseen delays in 
15 the presentation, during broadcast stoppage, and so on. 

Thus, an embodiment of the invention advantageously 
permits the user to view one or more substitute content 4 42 
on the television screen 444 during commercials and/or 
other breaks (e.g., broadcast stoppage) in a currently- 
20 shown live broadcast content or buffered broadcast content. 
The user can, therefore, optimize his/her viewing time by 
viewing other content when a break occurs in a currently- 
shown live broadcast content or buffered broadcast content. 
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An embodiment of the invention also advantageously 
enables an alternative medium for showing advertisements, 
since the substitute content 442 can also include 
advertisements . The advertisements in the substitute 
5 content 4 42 can be shown during commercial breaks of a 

currently shown broadcast content, during the skip segment 
(e.g., a 30-second skip segment) of a buffered broadcast 
where the skip segment contains recorded advertisements in 
the buffered broadcast, or during other breaks in a current 

10 broadcast content. 

The playback engine 430 may also have the 
functionality of a display engine for showing the images on 
the screen 444 (see Figure 5A) of the television 154 in 
various formats. For example, the substitute content 442 

15 may be displayed in full screen mode on the television 
screen 4 44, as shown in Figure 5A, while the current 
programming content is removed from the screen 444. 

Alternatively dr in addition, as shown in Figure 5B, 
the playback engine 430 may display the substitute content 
20 442 as the main screen 444 view, while displaying the live 
or other programming content 44 6 as a picture-in-pieture 
448 on the television screen 444. Of course, the 
substitute content 442 may alternatively be displayed in 
the picture-in-picture 4 48, while the programming content 
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448 is displayed as the main screen 444 view. 

• Alternatively or in addition, as shown in Figure 5C, 
the substitute content 4 42 may be displayed as a picture 
overlay on the television screen 444 foreground, while the 
5 live or other programming content 44 6 is displayed on the 
television screen 444 background. 

Alternatively or in addition, as shown in Figure 5D, 
the substitute content 4 42 and the live or other 
programming content 44 6 may be displayed in separate areas 
10 448 and 450, respectively, as, for example, split screen 
views on the television screen 444. 

Prior to sending the substitute content 4 42 from the 
hard disk 162 (or via a selected broadcast channel as 
graphically represented by line 322), the substitute 

15 content 4 42 is typically decoded by the converter 320 

{ Figure 3) prior to displaying the substitute content 442 
on the television screen 444. Similarly, prior to 
displaying the programming content 446 on the television 
screen 4 44, the programming content 44 6 is typically 

20 decoded by the converter 320. 

Referring again to Figure 4, the memory 330a may also 
store applications 4 45 which may represent one or more 
application. The applications 4'4 5 may also be stored in 
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the digital storage device 162. The applications 445 may 
include, for example, a browser for accessing a Uniform 
Resource Locator (URL) address on the Internet and for 
permitting web pages in a URL address to be displayed on 
5 the television screen 444. The applications 445 may also 
include an e-mail engine that can detect when an e-mail 
message has been received by the set top box 152a. * The 
email engine can also permit the display of an e-mail 
message on the television screen 444, the creation and 
10 transmission of an e-mail message from the STB 152a, and/or 
other e-mail processing functions. The applications 445 
may also include a streaming video engine for processing 
streaming video that is received from an IP channel. 

The applications 44 5 may also include middleware 
15 (sometimes called "plumbing") for connecting two sides of 
an application and passing data between them. 

The playback data 440 may be stored in, for example, 
the hard disk 162 and include a list of recorded substitute 
content 442 (e.g., substitute contents 442a and 442b. The 
20 playback engine 430 determines the user-selected substitute 
content 442 to be displayed on the television screen. 4 44 in 
response to a user-enabled command 426 (e.g., provided by a 
user via a remote control device) or in response to the 
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detection of a' break in the currently-shown programming 
content 44 6. 

The playback data 440 may also include a tag 451 
(Figure 4) identifying a specific recorded substitute 
5 content to be displayed as the default substitute content 
in response to user-enabled commands 426 or in response to 
the detection of a break in the currently-shown programming 
content 4 46. Thus, the default substitute content may also 
be displayed automatically when a commercial break is 
10 detected in the currently displayed programming content. 

Detection of commercial breaks in a programming content is 
performed by, for example, detecting identifiers that 
identify the beginning and end of a commercial. Various 
methods or mechanisms can be used to identify a sequence, 
15 such as, for example, include MPEG methods that multiplex 
special data stream (s) into the digital video transport 
stream, encoding/embedding data in the vertical blanking 
interval (VBI) portion of the analog television video 
signals (e.g., ATVEF triggers), using out-of-band modems 
20 (e.g., Digital Audio-Visual Council or "DAVIC" modems) to 
provide the interactive content, and other such techniques 
that provide information related to the content of the 
television signals. Another method to identify breaks in 
the programming content is by scanning the closed caption 
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text in the programming content for keywords that are of 
interest to the user. This method can be used to determine 
the beginning and ending of break segments. For example, a 
local new broadcast can be broken into "headlines", 
5 prominent local stories, national news, international news, 
business news, human interest stories, sports, weather, and 
commentary. A user may elect to skip the sports or weather 
section (especially an interactive user that receives these 
content on-demand) and to use this segment time to view 
10 buffered content (i.e., substitute content). 

Further, the playback data 4 40 may also include tags 
452 indicating the portions of a recorded substitute 
content that have been shown by the playback engine 430 for 
viewing on the television screen 44 4. The tags 4 52 thus 

15 permits the playback engine 430 to skip the showing of 

previously shown substitute content. Thus, as an example, 
the user may be viewing a first program (e.g., programming 
content 446) on the screen 444. When a commercial break 
occurs in the first program, the end user may initiate a 

20 second program (e.g., substitute content 442) to be 
displayed on the television screen 444. As discussed 
above, the user may reduce . the first program into, for 
example, a picture-in-picture 448 and bring the second 
program on the main window of screen 444. When the 
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commercial break in the first program comes to an end, the 
user can perform a swap function to bring the first program 
from the picture-in-picture 448 to the primary focus on the 
screen 444, In one embodiment, the second program will 
5 pause, and the tags 452 can be placed by the playback 
engine 430 (Figure 4) at the segment where the second 
program is paused. The user can then resume viewing of the 
second program (based on the tag 452 location) during the 
next set of commercials. Of course, other types of pause 
10 features or techniques may be used to indicate where the 
second program (substitute content) is paused. 

In one embodiment, a live programming content (such as 
content 446) may also be continuously recorded (e.g., in a 
live program buffer in hard disk 162) while the viewer is 

15 viewing the substitute content (e.g., substitute content 
4 42) . Thus, if a commercial break occurs in the live 
programming content, the viewer can switch viewing to the 
substitute content while the recording engine 410 records 
the live programming content. The live programming content 

20 may be, for example, recorded automatically (as programmed 
by the user) or upon receiving user-enabled commands from, 
for example, a user-controlled remote control device 158. 
If the commercial break (in the live programming content) 
ends before the viewer finishes watching at least a segment 
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of the substitute content 442, then the following 
compensation may occur. When the viewer switches viewing 
from the substitute content back to the programming content 
446, the user can, for example, issue via remote control 
5 device 158 a replay or rewind command so that the recorded 
portion of the live programming content is rewound to 
permit the user the opportunity to view the missed portions 
of the live programming content. Since the live 
programming content may be buffered in a live program 
10 buffer, the viewer can view the remaining segments of the 
live programming content after watching the missed recorded 
portions of the live programming content. Alternatively or 
in addition, the live program can be paused after the end 
of the commercial segment . 

15 Figure 6A is a block diagram of a programming content 

4 4 6a with a disable feature to prevent the disruption of 
viewing of commercial segments in the programming content 
4 46a. For example, an advertiser can pay the broadcaster 
extra fees (or rates) on the rate card to insert tags 455 

20 prior to a commercial segment 457. The tag 455 deactivates 
the playback engine 430 so that the viewer will not be able 
to view the substitute content 442 when the commercial 
segment 4 57 is showing on the television screen 444. As a 
result, an advertiser can pay extra rates to the 
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broadcaster to insure the viewing of the commercials of the 
advertisers to consumers . 

Figure 6B is a flowchart diagram illustrating a method 
4 60 of preventing the disruption of viewing of commercial 
5 segments in the programming content 44 6a. An advertiser or 
broadcaster (or other party) may pay (462) a multiple 
system operator (MSO) a fee or extra rate to prevent 
disruption of a commercial in the broadcast content. The 
fee or extra rate enables triggers or tags that prevent s 
10 subscriber to view substitute content during a commercial 
break in the broadcast content. The advertiser (or 
broadcaster or other party) transmits (4 64) a trigger that 
prevents the display of substitute content during a 
commercial break in the broadcast content. For example, 
15 the trigger or tag may prevent an overlay with substitute 
content from being displayed during the commercial break. 
As a result, the display of a substitute content is 
prevented (466) during a commercial break. The method 460 
permits the MSO to obtain additional revenues from the fee 
20 (or extra rate) paid by an advertiser (or broadcaster), and 
permits an advertiser to prevent disruptions of 
commercials . 

Figure 6C is a flowchart diagram illustrating a method 
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470 of permitting a subscriber to view substitute content 
during a commercial segment in the programming content 
446a. A subscriber may pay (472) a multiple system 
operator (MSO) a fee or extra rate to permit the subscriber 
5 to view substitute content during a commercial in the 

broadcast content. The advertiser (or broadcaster or other 
party) transmits (474) a trigger that prevents the display 
of substitute content during a commercial break in the 
broadcast content. For example, the trigger or tag may 

10 prevent an overlay with substitute content from being 

displayed during the commercial break. The subscriber can 
disable (47 6) the trigger or tag that prevents the display 
of substitute content. As a result , the subscriber may 
view (478) a substitute during a commercial break. The 

15 method 47 0 permits the MSO to obtain additional revenues 

from the fee (or extra rate) paid subscriber, and permits a 
subscriber to view substitute content during commercial 
breaks. 

Thus, the tags 455 (Figure 6A) may work in cooperation 
20 with an MSO and/or subscriber authorization features to 
enable/disable the effectiveness of the tags- 455. As 
mentioned above, a subscriber who pays an additional fee 
may be able to disable the. tags 4 55 so that the subscriber 
can view substitute content during breaks in the 
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programming content. An advertiser or broadcaster can pay 
(462) a fee or extra rates to an MSO to prevent the display 
of substitute content during a commercial break in a 
broadcast content or to prevent other disruptions of 
5 commercials in the broadcast content. 

Figure 7 is a flowchart diagram of a method 500 for 
substituting a substitute content in place of a live or 
pre-recorded programming content as shown on, for example, 
a television screen 444. In one embodiment, the playback 
10 engine 430 (Figure 4) may perform the method 500. The 
method 500 may run continuously or at representative 
intervals. Further, multiple instances of the method 500 
may run simultaneously. A first programming content is 
displayed (510) on, for example, the screen of the 
15 television 154 or other display device. The first 

programming content may be a live programming content, 
buffered programming content, or other programming content, 
such as a video telecast, teleconference, or prerecorded 
programming content. Next, a substitute signal is received 
20 (520) . The substitute signal may be received by the STB 
152a from, for example, a remote control device or other 
device, such as the remote control device 158. Typically, 
a viewer will send the substitute signal via the remote 
control device 158 upon viewing a commercial on the 
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television 154. However, the viewer may also send the 
substitute signal at any time, such as when live action 
takes a break between speakers in a teleconference. In 
another embodiment of the invention, the substitute signal 
5 is a tag in the broadcast, indicating the start of a 
commercial . 

In one embodiment, the substitute signal may be 
ignored if a commercial contains a signal, such as a 
particular tag, indicating that the viewing of the 
10 commercial in the programming content should not be 

disrupted. For example, this enables an advertiser, upon 
payment of an extra fee, to prevent skipping of commercials 
or substitution over commercials . 

Next, the substitute content is displayed (530) . In 
15 an embodiment of the invention, the playback engine 430 
selects the substitute content from the hard disk 162 for 
viewing based on the playback data 440. For example, if 
there is multiple substitute content on the hard disk 162, 
then the playback engine 4 30 can select a default 
20 programming content or pre-specif ied substitute content as 
indicated , in. data 440. -Further, if a portion of the 
substitute content has already been viewed, as indicated by 
a tag (tags 452a & 542b in Figure 4) in the data 440, then 
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the playback engine 430 will begin displaying the 
substitute content starting from the end of the previously 
viewed content portion- In another embodiment of the 
invention, the playback engine 4 30 enables a viewer to 
5 select a substitute content to view from a list of 

substitute content in the playback data 44 0 upon receipt of 
a substitute signal. In another embodiment, the playback 
engine 430 selects substitute content including a live 
program that is transmitted along a specified channel. In 
10 another embodiment, the playback engine 430 selects 

substitute content including IP channel content such as 
streaming video, e-mail interaction, and so on. The IP 
channel content may be received by the STB 152a, for 
example, via the cable net 134. 

15 While the substitute content is being displayed (530), 

a small window may also be optionally displayed (540) 
showing a reduced view of the first programming content, 
thus enabling a viewer to monitor the status of the first 
programming content. For example, if a viewer sends a 

20 substitute signal to avoid watching commercials, the viewer 
will want to return to view the first programming content 
after the end of the commercials. In one embodiment, the 
first programming content may also be continuously recorded 
by recording engine 410 while viewing the substitute 
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content. Of course, the first programming content may also 
be completely removed from the screen when the substitute 
content is being shown on the screen. 

In another embodiment, only the replayed segment of a 
5 sports broadcast are recorded by recording engine 410 or 
tagged by playback engine 4 30. To determine the replayed 
segment, various methods may be used such as detecting for 
a signal difference that characterize the replayed segment, 
or comparing the replayed segment with a prior recorded 
10 content stored in a buffer. Typically, many people are 
probably only interested in highlights (e.g., great plays 
in a baseball game) which are replayed in, for example, a 
sports news program. 

In another embodiment, the close caption of a live 
15 programming content (e.g., a sports news show) is watched 
by the viewer while the viewer is watching a pre-recorded 
or streaming content to view a highlight (e.g., a great 
play in a baseball game) . 

In another embodiment of the invention, a countdown 
20 timer may be displayed in addition to or as an alternative 
to the small window. The countdown timer may count down in 
30-second increments, indicating the end portion of 
successive 30-second commercials. The viewer can then 



41 



BNSOOCID: <WO 02104037A1_IA> 



WO 02/104037 



PCT7US02/09148 



pause or halt the substitute content and continue viewing 
the first programming content on the screen. 

Next, an end-substitution signal is received (550) . 
In an embodiment of the invention, the remote control 
device 158 can send the end-substitution signal to the STB 
152a. In another embodiment of the invention, a tag in the 
programming content may indicate the end of a commercial or 
the restart of the non-commercial segment in the 
programming content. Tags 4 52a and 452b may then be stored 
(560) in, for example, the playback data 440, marking the 
portion of the substitute content that has been viewed. In 
an embodiment of the invention, the playback engine 430 
stores the tags 452a and 452b. The first programming 
content is then displayed (570) in full screen on the 
television 154 by, for example, the playback engine 430. 
In addition, the substitute content may be displayed in a 
small window in reduced size and in a paused mode. 

Figure 8 illustrates a block diagram of another 
representative set top box (or other customer premise 
equipment or processing device) according to a specific 
embodiment of the invention. The STB 152b includes a 
network interface 300, processor 310, memory device 330b, 
transceiver 157, converter 320, cable modem 156, clock 350, 
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and hard disk 162 (also referred to as storage device 162), 
all interconnected together for communication via system 
bus 340. In one embodiment, the components of STB 152b, 
except for at least some of the programming content in the 
5 memory device 330b, may be substantially similar to 

corresponding components in the STB 152a. The processor 
(controller) 310 executes instructions stored in the memory 
330b, which will be discussed in further detail in 
conjunction with Figure 10. 

10 Figure 9 is a block diagram of a memory device 330b of 

the STB 152b (Figure 8). In one embodiment, the memory 
device 330b includes a trigger marking engine 710, marked 
triggers 720, and trigger viewing engine 730. Of course, 
other applications, such as the applications 445 in Figure 

15 4, may also be loaded in the memory 330b. Additionally, 
the engines 710, 720, and 730 may also be loaded into the 
memory 330a (Figure 4) and may enable the below described 
functionalities in the STB 152a. 

The trigger marking engine 710 enables a viewer to 
20 mark triggers (in a broadcast content) during television 
viewing for follow up viewing.. _In one embodiment , . the 
trigger marking engine 710 stores the marked triggers in 
the marked triggers 720 where a marked trigger includes, 
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for example, a URL address (or other address information) 
associated with the marked trigger. The trigger viewing 
engine 730 enables a viewer to later select a marked 
trigger and access a URL address (or other address 

5 information) associated with the selected marked trigger. 
Thus, when, for example, a break occurs in a live 
programming content (or buffered programming content, 
recorded programming content, or other content), a viewer 
can select and access a URL address in order to access the 

10 website (or other interactive site) that is linked to the 
URL address. Alternatively, the user may access other 
information linked to the selected marked trigger. The 
website can be shown on the television screen 444 in 
various formats as similarly shown, for example, by the 

15 substitute content 442 in Figures 5A, 5B, 5C, and/or 5D, 
while the programming content may be removed from the 
screen 444 or shown as in Figures 5B, 5C, and 5D during the 
break in the broadcast programming content. Thus, during 
the break, the viewer can view information and/or buy 

20 products or services from the displayed website (or other 
displayed information) . 

Figure 10 is a flowchart diagram of a method 800 for 
substituting live programming content or other programming 
content with trigger-associated information. In one 
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embodiment, the trigger marking engine 710 in conjunction 
•with the trigger viewing engine 7 30 performs the operation 
of method 800. The method 800 may run continuously or at 
representative intervals. Further, multiple instances of 
5 the method 800 may be run simultaneously on a single STB. 

First, a programming content is displayed (810) on, 
for example, the television screen 444. The programming 
content may include a live programming content or pre- 
recorded or buffered programming content. The programming 

10 content also includes triggers , which may include 

associated URL or other address information. The URL or 
other address information is simultaneously displayed on 
the television screen 44 4 with the programming content. If 
a viewer of the programming content is interested in the 

15 trigger, the viewer can mark the trigger by, for example, 
sending a command via a remote control device, such as 
device 158, to the STB 152b, where the command is received 
(820) . In a single viewing session, the viewer may mark a 
plurality of triggers. 

20 The marked triggers are then stored (830) . In one 

embodiment, the ..trigger marking, engine 710 stores, the 
marked* triggers in marked triggers 720 which may be URL or 
other address information. Next, a skip-program signal is 
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received (840). If there is a single stored marked 
trigger, then information associated with the single 
trigger is displayed (850) . For example, if the trigger is 
a website address, the website is loaded and displayed. In 
5 one embodiment, the website or information associated with 
the trigger is preloaded by the playback engine 4 30 so that 
when the website address is selected, the viewer during the 
commercial break does not need to wait for the website or 
information to load. If there are multiple stored 
10 triggers, then information associated with the triggers may 
be cycled after a pre-defined amount of time or according 
to viewer input. Further, a viewer may be able to select 
which trigger information to display from list of stored 
triggers. In an embodiment of the invention, the trigger 
15 viewing engine 730 displays the information associated with 
a trigger. 

In addition to displaying information associated with 
a trigger, the programming content may also be optionally 
displayed on the television screen 4 44. Alternatively or 
20 in addition to displaying the programming content, a 

countdown timer may be displayed to indicate to the viewer 
when a commercial period ends. The timer may countdown in 
30-second increments or other increments. 
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An end-skip signal is then received (860). In one 
embodiment of the invention, the remote control device 158 
may send the end-skip signal, which is then received by STB 
152b. Triggers, for which trigger-associated information 
5 has been displayed, may then be deleted (.870) . In one 

embodiment of the invention, the trigger viewing engine 730 
deletes displayed triggers from the marked triggers 720. 
The programming content is then displayed (880) in full 
screen mode on the screen 4 44. 

10 

Aggregation of Substitute Content 

Referring now to Figure 11A, shown generally at 900 is 
an interactive television system that can implement another 
specific embodiment of the invention. The system 900 

15 includes one or more head-ends 902 (or broadcast centers) 
to distribute television broadcast content and interactive 
television (sometimes referred to as M iTV") content to a 
plurality of subscriber residences 904. The head-end 902 
includes broadcast content processing and iTV services 

20 hardware and software (including processors, servers, 
storage media, databases, and so on) , which are shown 
generally in Figure 11A as a unit 906. .The unit 906 can 
receive national broadcast content and iTV content 908 from 
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content providers, satellite feeds, servers, and/or other 
sources. The unit 906 can also receive local broadcast 
content and iTV content 910 from parties such as local 
studios , servers, content providers, and/or other sources. 
5 A cable modem termination system (CMTS) 912 provides 

the head-end 902 (and ultimately the subscriber residences 
904) with access to the Internet 914. A combiner 916 
places the signals and/or data provided by the unit 906 and 
the CMTS 912 into appropriate channels of a communication 
10 network 918 . In accordance with an embodiment of the 

invention, the communication network 918 can include, for 
example, a hybrid fiber-coaxial cable distribution network 
comprising channels having a bandwidth of approximately six 
(6) MHz. Some of these channels of the communication 
15 network 918 are used to carry television signals (which may 
be accompanied by interactive content and other trigger 
information) , while other channels are used for upstream 
and downstream communication of data with the Internet 914 . 
Some other channels (sometimes referred to as "out-of-band" 
20 channels) of the communication network 918 may be used to 
carry electronic program guide (EPG) information. In one 
embodiment, at least one of these out-of-band channels may 
be used to carry aggregated television programming content 
and/or related data. 
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Examples of techniques that can be used to provide 
interactive content and other trigger information from the 
unit 906 to set top boxes (or other types of customer 
premise equipment) in the subscriber residences 904, via 
5 channels of the communication network 918, include MPEG 
methods that multiplex special data stream (s) into the 
digital video transport stream, encoding/embedding data in 
the vertical blanking interval (VBI) portion of the analog 
television video signals (e.g., ATVEF triggers), using out- 

10 of -band modems (e.g., Digital Audio-Visual Council or 
"DAVIC" modems) to provide the interactive content, and 
other such techniques that provide information related to 
the content of the television signals. For the channels of 
the communication network 918 that are used for 

15 communication of data to and from the Internet 914, 
protocols such as Data Over Cable Service Interface 
Specifications (DOCSIS), transmission control 
protocol/Internet protocol (TCP/IP), hypertext transfer 
protocol (HTTP), file transfer protocol (FTP), or other 

20 suitable protocols/techniques may be used. 

In an embodiment of the invention, ATVEF triggers or 
other triggering mechanisms can be used to identify 
specific topics of particular television programming 
content in a television signal. For instance, if the 
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sports segment of a news telecast is showing highlights of 
a particular baseball player's homerun, that baseball 
player can be identified in the trigger. As another 
example, triggers may be used to demarcate the beginning 
and ending locations of that baseball player's highlights, 
the sports segment, the commercial breaks of the news 
telecast, or the news telecast itself. 

In an embodiment of the invention, an aggregator 94 0 
can aggregate user-specified television programming 
content. The aggregator 94 0 can include a combination of 
hardware and software (such as one or more processors, one 
or more servers, databases and/or other storage media, 
software and other machine-readable instructions stored on 
a machine-readable medium, and so on) that delivers its 
aggregated content to client terminals of subscribers via 
at least one of the channels of the communication network 
918, or via some other communication link (such as a DSL 
connection or telephone modem connection) . The aggregator 
94 0 can be located in the head-end 902 in one embodiment, 
while in other embodiments, the aggregator 940 can be 
located outside of the head-end 902, including somewhere in 
the Internet 914. 

The aggregator 940 can be communicatively coupled to 
the unit 906 and/or to the CMTS 912. For instance, in 

50 



<W Q 021 04037A 1 _IA> 



WO 02/104037 PCT/US02/09148 



several example implementations where the aggregator 94 0 
operates to provide URL addresses (or other interactive 
information peculiar to the Internet 914 or other suitable 
data communication network) to the client terminals of 
5 subscribers, the aggregator 940 can be communicatively 

coupled to the CMTS 912. It is to be appreciated that the 
aggregator 94 0 need not necessarily be coupled to the CMTS 
912 in some embodiments, provided that the aggregator 94 0 
can aggregate television programming content from various 
10 sources and then deliver the aggregated television 
programming content to subscribers. 

As mentioned, the aggregator 940 can be coupled to or 
comprise part of the unit 906, as depicted in Figure 11A. 
In such an embodiment, the aggregator 940 can have access 
15 to a database 905 that stores user-specified preferences 
for programming content as provided by a viewer. For 
instance, entries in the database 905 may indicate that the 
particular viewer wishes to aggregate all highlights for a 
particular baseball player for a specified time period. 
20 The database 905 may further identify the viewer by 

residence address, identification code of the viewer's 
client terminal (e.g., a set top box or recording device 
having network connectivity), or other information to 
uniquely identify the viewer from other viewers. Then, the 
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aggregator 94 0 can monitor television signals provided from 
sources 908 and 910 to the unit 906 for that baseball 
player' s highlights for the specified time period, and then 
store the highlights in a storage unit (e.g., database 905) 
at the head-end 902 . 

Various techniques may be used by the aggregator 940 
to locate the user-specified television programming 
content. In one embodiment, the aggregator 940 can read 
triggers of all television signals that arrive at the unit 
906, so as to identify the topics /content of the television 
signals as specified in the triggers, and then correlate 
the trigger content with the entries in the database 905. 
If relevant television programming content segments are 
detected from the triggers, then the aggregator 94 0 records 
such segments. In another embodiment, the aggregator 940 
can identify television programming content segments of 
interest by reading close-captioning data that accompanies 
the television signals. Alternatively or in addition, the 
aggregator 940 can scan EPG information that is provided to 
the head-end 902, with the EPG information specifying data 
such as show titles, actors, subjects, and the like. 

Various other techniques can be used by the aggregator 
940 to aggregate television programming content and to 
provide the aggregated television programming content to 
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subscribers. Examples of these techniques include 
receiving the television programming content from third- 
party entities that collect the programming content as a • 
service, or receiving the television programming content 
5 via direct feeds from content providers to the aggregator 
940. 

At the residence 904, a splitter 930 is coupled to 
receive the television signals (including interactive 
content and other trigger information) from the 

10 communication network 918. The splitter 930 sends the 

received television signals (including interactive content 
and other trigger information) to a first client terminal, 
such as a set top box 152, which subsequently sends the 
signals and other information to a television 154. As will 

15 be understood by those skilled in the art, the set top box 
152 includes components such as digital/analog video 
processing unit(s), VBI decoders, central processing units, 
memory, video graphics processing unit(s), software, and 
other components that can detect, extract, and process 

20 trigger information and other interactive content that is 

received from the communication network 918. For instance, 
these components can extract URL addresses from the trigger 
information and present the URL addresses as hypertext 
links or other visual indicator on the television 154, 
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which the viewer can x> click" to connect to the Internet 914 
(or other suitable data communication network) . The 
operation of these components is generally known to those 
of ordinary skill in the art. Accordingly, the specifics 
5 of their operation are not shown in Figure 11A or described 
in further detail herein for the sake of simplicity of 
explanation - 

As previously stated above, the set top box 152 can 
include or be coupled to the storage unit 162. The storage 

10 unit 162 can comprise a machine-readable storage medium 
such as a cache, buffer, memory, diskette, compact disk, 
tape, flash memory, random access memory (RAM) or the like 
and their associated hardware, in one embodiment. In an 
embodiment the storage unit 162 can comprise a hard disk 

15 such as a digital or personal video recorder (DVR or PVR) . 

In some embodiments, the splitter 930 can be coupled 
to the cable modem 15 6. With basic embodiments of the 
splitter 930, the cable modem 156 receives the same input 
as the set top box 152 (e.g., receives the same television 

20 signals) . With more complicated embodiments of the 

splitter 930, the cable modem 156 and the set top box 152 
may receive different channels. Whether receiving the same 
or different channels from the splitter 930, a primary 
function of the cable modem 156 is to provide one or more 
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ancillary devices 938 with access to the Internet 914. 
Examples of the ancillary device 938 can include, but not 
be limited to, Personal Computers (PCs), wireless handheld 
devices, companion set top boxes, two-way advanced remote 
5 controls, or other browser-enabled display devices or 

client terminals that have the capability to communicate 
with the Internet 914. The ancillary device (s) 938 can 
include or be coupled to the storage unit 162 (or other 
storage unit) . 

10 In operation according to an embodiment, aggregated 

television programming content can be sent to the storage 
unit 162 (or to the memory 330 in the STB 152) by the 
aggregator 940. This may be done by way of broadcast, IP 
multicast, or unicast from the aggregator 940 to storage 

15 unit(s) 162 of one or more subscriber residences 904. In 
accordance with an embodiment of the invention, the 
aggregated television programming content can be sent to 
the storage unit 162 by first identifying the storage unit 
that is to receive the aggregated television programming 

20 content, such as via an identification number of the 
storage unit 162, and then sending the aggregated 
television programming content to the storage unit 162 in a 
format that is compatible with the particular communication 
network and with the storage unit 162 (such as a digital 
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format) . For example, the aggregated television program 
can be sent to the storage unit 162 by way of the CMTS 912 , 
through a channel of the communication network 918 that 
uses the DOCSIS protocol, and then to the cable modem 156. 
5 Alternatively or in addition, an out-of-band channel may be 
used to send the aggregated television programming content 
from the aggregator 940. In yet other embodiments, the 
aggregated television programming content may be sent to 
the storage unit 162 via a different connection to the 
10 Internet 914, or the aggregated television programming 

content may be sent via one of the available channels of 
the communication network 918. 

Afterwards, incoming real-time television programming 
content (or buffered incoming programming content) is 
15 monitored by the set top box 152 for triggers that 

demarcate commercial breaks. In accordance with one 
embodiment of the invention when a trigger is received to 
indicate the beginning of a commercial break in the real- 
time television programming content, the television 154 is 
20 coupled to the storage unit 162 for playback of the stored 
programming content during the commercial break. When a 
trigger is received to indicate the end of the commercial 
break, the input to the television 154 is switched from the 
storage unit 162 back to the real-time programming content. 
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During presentation of the real-time television programming 
content, a pause feature of the storage unit 162 may be 
used to pause or otherwise mark the stored television 
programming content, so that when it resumes playing during 
5 a subsequent commercial, it can begin playback at the 
proper location. 

As similarly described above, the stored programming 
content (e.g., substitute content) 442 may also be 
displayed to a viewer during a break in the incoming 

10 television program (or during a break in the buffered 

incoming television program) by having the viewer enable a 
command 42 6 for displaying the stored programming content. 

In accordance with another embodiment of the 
invention, the aggregated television programming content 

15 may be delivered for presentation by way of switching to a 
channel where the programming content is carried. For 
example, the aggregator 940 may first store the aggregated 
television programming content in the storage unit 905 at 
the head-end 902. Upon recognition of a trigger that 

20 demarcates the beginning of a commercial break in a real- 
time television programming content, the set top box 152 
can send a signal to the unit 906 at the head-end 902 to 
instruct the unit 906 to provide the aggregated television 
programming content. The unit 906 then sends a command 
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back to the set top box 152 that instructs the set top box 
152 to tune/switch from the channel of the real-time 
programming content (e.g., channel 945) to a different 
channel (i.e., a dedicated broadcast channel 947) where the 
aggregated programming content is to be carried. The 
aggregated television programming content is then delivered 
on that dedicated broadcast channel 947 to the set top box 
152 by the unit 906. The process can work similarly (in 
reverse) when the set top box 152 recognizes a trigger that 
demarcates the end of the commercial break, resulting in a 
switch back to the original channel 945 and the pausing of 
the aggregated television programming content. 

The set top box 152 can also tune/switch from the 
original channel 945 to the dedicated broadcast channel 947 
(and tune/switch from the dedicated broadcast channel 947 
to the original channel 945) by having the viewer enable 
appropriate commands 960. 

In another embodiment of the invention, aggregator 940 
directs commands 965 to STB 152 to tune to a particular 
channel (e.g., channel 947) at a particular time based on 
user-specified television programming content preferences. 
The aggregator 940 may also command STB 152 to begin 
recording and end recording so as to only record the 
programming content corresponding to the user preferences. 
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The programming content may be recorded in, for example, 
the storage medium 162. At a later time, aggregator 940 
can then command STB 152 to substitute a live programming 
content (or buffered incoming programming content) with the 
5 recorded programming content. The advantages of recording 
locally at the STB 152 may include the following: (1) the 
programming content does not have to be captured and 
recorded at the aggregator 940 and then transmitted to the 
STB 152, and (2) dedicated bandwidth for transmission of 

10 the aggregated programming content is not required. In 
this embodiment, preferably, the STB 152 will ignore 
commands received from aggregator 940 if STB 152 is ' 
currently tuned to a channel, indicating that the STB is in 
use. If the STB' 152 has dual channel tuning capability and 

15 dual channel recording capability, then STB 152 may display 
the programming content from a first channel and still 
record from another channel as specified by the aggregator 
940. 

In the above-embodiment, if the STB 152 has only 
20 single channel tuning and single channel recording 

capability, then programming content aggregation can be 
accomplished during periods in which the STB 152 is not 
being used (e.g., during late at night or when the user is 
on vacation) . This is because the only available resource 
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is utilized during regular television viewing. If the STB 
152 has dual channel tuning and dual channel recording 
capability, then one channel can be used for the 
aggregation of programming content, while another channel 
can be used for regular television viewing. 

It is to be appreciated that in other embodiments, the 
unit 90 6 can be capable of detecting triggers that 
demarcate the beginning and ending of commercial breaks. 
In such embodiments, the unit 906 can send a command 970 to 
the set top box 152 to tune to the channel where the 
aggregated television programming content is to be carried, 
instead of having the set top box 152 send a signal to the 
unit 906 that requests playback of the aggregated 
television programming content. 

In another embodiment as shown in Figure 12, various 
subscriber areas 904a to 904c can communicate with each 
other via the Internet 914 or other data communication 
network. The number of subscriber areas 904 may vary. A 
server 980 may include an application 982 that permits the 
following functions. For example, a user at the area 904a 
may be able to access directories in the memory 330b or 
hard drive 162b (in area 904b) or directories in the memory 
330c or hard drive 162c (in area 904c) to a file-sharing 
system via the Internet or network 914. The user in area 
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904a may then view and download the aggregated content in 
the accessed directories of the above-mentioned memory 
devices- The downloaded aggregated content may be stored 
in the memory devices 330a and/or 162a and displayed during 
5 a break in a programming content that is being received by 
the STB 1512a. 

Similarly, a user in the area 904b may be able to 
access directories of memory devices in areas 904a and 904c 
and download aggregated content that are stored in those 
10 memory devices. ^ 

Figure 12 is a flowchart of a method 1000 for 
collecting and displaying aggregated substitute content in 
accordance with a specific embodiment of the invention. 
The method 1000 first aggregates (1005) substitute content 

15 based upon user or viewer preference (s) . The aggregate 

content may then be delivered (1010) to a customer premise 
equipment (e.g., set top box 152) of the viewer. During a 
break in a currently-shown content, at least one of the 
substitute content in the aggregate content may be 

20 displayed (1015) . When the break in the currently-shown 
content terminates, the display of the- substitute -content 
may be paused or halted (1020) . The currently-shown 
content may continue to be displayed. 
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Other variations and modifications of the above- 
described embodiments and methods are possible in light of 
the foregoing teaching. For example, the components, 
modules and software shown in Figures 3 and 4 may be 
5 implemented in a processing device such as a personal 
computer or a suitable notebook computer. 

Further, at least some of the components of this 
invention may be implemented by using a programmed general 
purpose digital computer, by using application specific 
10 integrated circuits (ASICs) , programmable logic devices 
(PLDs) or field programmable gate arrays (FPGAs) , or by 
using a network of interconnected components and circuits. 
Connections may be wired, wireless, by modem, and the like. 

It is also within the scope of the present invention to 
15 implement a program or code that can be stored in an 
electronically-readable medium to permit a computer to 
perform any of the methods described above. 

The above description of illustrated embodiments of 
the invention, including what is described in the Abstract, 
20 is not intended to be exhaustive or to limit the invention 
to the precise forms disclosed. While specific embodiments 
of, and examples for, the invention are described herein 
for illustrative purposes, various equivalent modifications 
are possible within the scope of the invention, as those 
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skilled in the relevant art will recognize. 

These modifications can be made to the invention in 
light of the above detailed description. The terms used in 
the following claims should not be construed to limit the 
invention to the specific embodiments disclosed in the 
specification and the claims. Rather, the scope of the 
invention is to be determined entirely by the following 
claims, which are to be construed in accordance with 
established doctrines of claim interpretation. 
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CLAIMS 

WHAT IS CLAIMED IS : 

1 . An apparatus for performing a substitution for a 
broadcast content, the apparatus comprising: 

5 a substitute content engine configured to detect a 

break in a broadcast content and to show substitute content 
in response to the break in a broadcast content; and 

a processor configured to execute the substitute 
content engine. 

10 

2. The apparatus of claim 1 wherein the substitute 
content engine is configured to record substitute content. 

3. The apparatus of claim 1 wherein the substitute 
15 content engine is configured to display the broadcast 

content when the break terminates. 

4 . The apparatus of claim 1 wherein the broadcast content 
comprises live programming content. 

20 

5. The apparatus of claim 1 wherein the broadcast content 
comprises buffered programming content. 

6. The apparatus of claim 1 wherein the broadcast content 
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5 



comprises video telecast content. 

7. The apparatus of claim 1 wherein the broadcast content 
comprises teleconference content. 

8. The apparatus of claim 7 wherein the teleconference 
content comprises remote learning session broadcast 
content. 



10 9. The apparatus of claim 7 wherein the teleconference 
content comprises video conference content. 

10. The apparatus of claim 1 wherein the break in the 
broadcast content includes a commercial segment. 

15 

11. The apparatus of claim 1 wherein the break in the 
broadcast content includes a commercial skip segment. 

12. The apparatus of claim 1 wherein the break in the 
20 broadcast content represents a broadcast stoppage. 

13. The apparatus of claim 1 wherein the break in the 
broadcast content represents non-activity in a broadcast 
event . 
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14. The apparatus of claim 1 wherein the substitute 
content comprises live broadcast content. 

15. The apparatus of claim 1 wherein the substitute 
content comprises buffered broadcast content. 

16. The apparatus of claim 1 wherein the substitute 
content comprises recorded broadcast content. 

17. The apparatus of claim 1 wherein the substitute 
content comprises Internet Protocol channel content. 

18. The apparatus of claim 17 wherein the Internet 
Protocol channel content comprises streaming video. 

19. The apparatus of claim 17 wherein the Internet 
Protocol channel content comprises electronic mail 
interaction . 

20. The apparatus of claim 1 wherein the substitute 
content is displayed in full screen mode. 

21. The apparatus of claim 1 wherein the substitute 
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content is displayed as a main screen view and the 
broadcast content is displayed as a picture-in-picture . 

22. The apparatus of claim 1 wherein the substitute 
5 content is displayed in the screen foreground and the 

broadcast content is displayed in the screen background. 

23. The apparatus of claim 1 wherein the substitute 
content is displayed in a first screen area and the 

10 broadcast content is displayed in a second screen area. 

24. The apparatus of claim 1 wherein the substitute 

. content is displayed in response to a user-enabled command. 

15 25. The apparatus of claim 1 wherein the substitute 

content is automatically displayed upon detection of the 
break in the broadcast content. 

26. The apparatus of claim 1 wherein the viewing engine is 
20 capable to display a timer to indicate a time length for 

showing the substitute content. 

27. The apparatus of claim 1 wherein the substitute 
content is automatically displayed in response to the break 
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in the broadcast content. 

28. The apparatus of claim 1 wherein the viewing engine is 
capable to pause the substitute content when the break in 

5 the broadcast content terminates. 

29. The apparatus of claim 1 wherein the substitute 
content engine is configured to record the broadcast 
content when the substitute content is displayed. 

10 

30. The apparatus of claim 1 wherein the substitute 
content engine is configured to pause the broadcast content 
when the break ends. 

15 31. The apparatus of ciaim 1 wherein the broadcast content 
includes a tag to prevent disruption of the display of the 
broadcast content during the occurrence of the break. 

32. The apparatus of claim 31 wherein a first party pays 
20 an operator to include the tag in the broadcast content. 

33. The apparatus of claim 31 wherein a subscriber pays an 
operator to disable the tag. 
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34. The apparatus of claim 1 wherein the broadcast 'content 
includes a trigger. 

35. The apparatus of claim 1, wherein the substitute 
5 content is associated with the trigger. 

36. The apparatus of claim 35 wherein the trigger is 
associated with an information site. 

10 37. The apparatus of claim 36 wherein the information site 
is loaded prior to a break in the broadcast content. 

38. The apparatus of claim. 1, further comprising: 

an aggregator communicatively coupled to the customer 
15 premise equipment and capable to aggregate at least one 
substitute content based upon a viewer preference. 

39. The apparatus of claim 1 wherein the broadcast content 
is recorded to permit viewing of a missed segment in the 

20 broadcast content when the display of the substitute 
content terminates . 

40. The apparatus of claim 1 wherein a pre-selected 
segment of the broadcast content is recorded during display 
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of the substitute content, 

41. The apparatus of claim 40 wherein the pre-selected 
segment is content replayed during the showing of the 

5 broadcast content. 

42. The apparatus of claim 40 wherein the pre-segment is 
viewed while displaying close caption text from the 
broadcast content . 

10 

43. An apparatus for substituting broadcast content 
displayed on a screen, the apparatus comprising: 

a customer premise equipment capable to receive 
broadcast content; 
15 the customer premise equipment including: 

a marking engine capable to mark at least one trigger 
in a broadcast content; and 

a viewing engine communicatively coupled to the 
marking engine and capable to permit the display of 
20 information associated with a selected marked trigger on 
the screen. 

44. The apparatus of claim 4 3 wherein the displayed 
information is an interactive site. 
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45. The apparatus of claim 4 4 wherein the interactive site 
can receive a purchase or request from the viewer. 

5 46. An apparatus for substituting broadcast content 
displayed on a screen, the apparatus comprising: 

means for determining an occurrence of a break in a 
broadcast content; and 

coupled to the determining means, means for displaying 
10 substitute content on the screen in response to the 
occurrence of the break. 

47. An apparatus for substituting broadcast content 
displayed on a screen, the apparatus comprising: 
15 means for marking at least one trigger in a broadcast 

content; 

coupled to the marking means, means for determining an 
occurrence of a break in the broadcast content; and 

coupled to the marking means, means for displaying 
20 information associated with a selected marked trigger on 
the screen in response to the occurrence of the break. 



48. A system for preventing the disruption of break 
segments in a broadcast content, the system comprising: 
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a first stage configured to insert a trigger in a 
broadcast content, the trigger configured to prevent the 
disruption of a break in the broadcast content; 

a transmission path configured to transmit the trigger 
5 from the first stage; and 

a destination stage configured to receive the trigger 
along the transmission path and to prevent the disruption 
of the break in the broadcast content. 

10 49. The system of claim 4 8 wherein the trigger is inserted 
in the broadcast content based upon payment of a fee. 

50. A system for permitting the display of content during 
break segments in a broadcast content, the system 
15 comprising: 

a first stage configured to insert a trigger in a 
broadcast content, the trigger configured to prevent the 
disruption of a break in the broadcast content; 

a transmission path configured to transmit the trigger 
20 from the first stage; and 

a destination stage configured to receive the trigger 
along the transmission path and to disable the trigger to 
permit display of substitute content during the break in 
the broadcast content. 
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51. The system of claim 50 wherein the trigger can be 
disabled based upon payment of a fee. 

5 52. An apparatus for substituting broadcast content 
displayed on a screen, the apparatus comprising: 

a customer premise equipment capable to receive 
broadcast content ; 

the customer premise equipment including a viewing 
10 engine capable to permit the display of substitute content 
in response to a break in the broadcast content. 

53. The apparatus of claim 52 wherein the broadcast 
content is displayed when the break terminates. 

15 

54. The apparatus of claim 52 wherein the broadcast 
content comprises live programming content. 

55. The apparatus of claim 52 wherein the broadcast 
20 content comprises buffered programming content. 



56. The apparatus of claim 52 wherein the broadcast 
content comprises video telecast content. 
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57 . The apparatus of claim 52 wherein the broadcast 
content comprises teleconference content. 

58. The apparatus of claim 57 wherein the teleconference 
content comprises remote learning session broadcast 
content . 

59. The apparatus of claim 57 wherein the teleconference 
content comprises video conference content. 

60. The apparatus of claim 52 wherein the break in the 
broadcast content includes a commercial segment. 

61. The apparatus of claim 52 wherein the break in the 
broadcast content includes a commercial skip segment. 

62. The apparatus of claim 52 wherein the break in the 
broadcast content represents a broadcast stoppage. 

63. The apparatus of claim 52 wherein the break in the 
broadcast content represents non-activity in a broadcast 
event . 

64 . The apparatus of claim 52 wherein the substitute 
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content comprises live broadcast content, 

65. The apparatus of claim 52 wherein the substitute 
content comprises buffered broadcast content. 

5 

66. The apparatus of claim 52 wherein the substitute 
content comprises recorded broadcast content. 

67. The apparatus of claim 52 wherein the substitute 
10 content comprises Internet Protocol channel content. 

68. The apparatus of claim 67 wherein the Internet 
Protocol channel content comprises streaming video. 

15 69. The apparatus of claim 67 wherein the Internet 
Protocol channel content comprises electronic mail 
interaction. 

70, The apparatus of claim 52 wherein the substitute 
20 content is displayed in full screen mode. 

71. The apparatus of claim 52 wherein the substitute 
content is displayed as a main screen view and the 
broadcast content is displayed as a picture-in-picture . 
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72. The apparatus of claim 52 wherein the substitute 
content is displayed in the screen foreground and the 
broadcast content is displayed in the screen background. 

5 

73. The apparatus of claim 52 wherein the substitute 
content is displayed in a first screen area and the 
broadcast content is displayed in a second screen area. 

10 74. The apparatus of claim 52 wherein the substitute 

content is displayed in response to a user-enabled command. 

75. The apparatus of claim 52 wherein the substitute 
content is automatically displayed upon detection of the 

15 break in the broadcast content. 

76. The apparatus of claim 52 wherein the viewing engine 
is capable to display a timer to indicate a time length for 
showing the substitute content. 
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